Distlog

The distlog package is the library you add to your code. It is used in combination with the regular Python logging module.

distlog.import_task(_id, msg, *args, **kwargs)

Link task to external parent.

Let this task be a subtask of a task in a different process. Allows you to see the sequence of calls and log messages over process boundaries.

_id is obtained in the calling process through the get_foreign_task(). How the value is transferred from the caller to the callee is application defined. That depends on the application and communication protocols.

from distlog import import_task
id = get_foreign_task_id()
myhost = 'sample.example.com'

with import_task(id, 'continued processing', host=myhost):
    # do something interesting

Parameters:

• _id (string) – task id calculated by the foreign parent task
• msg (string) – log message used with entering (and optionally when leaving the task
• args (list) – parameters for the log message
• kwargs (dict) – key/value pairs forming the log message context.

Return type:

Task

distlog.task(msg, *args, **kwargs)

Create a toplevel task.

Creates a new toplevel context. You would use this when a new activity is started. A task is generally composed from smaller subtasks.

from distlog import task
with task('counting up to %d', 10, job='count up', until=10):
    for i in range(10):
        print(i)

Parameters:

• msg (string) – state the goal of this program
• args (list) – parameters for the goal
• kwargs (dict) – key/value context for the log messages

Return type:

Task

distlog.to(msg, *args, **kwargs)

Create a subtask.

Encapsulates a part of a larger program. You typically use this to delineate a program section that performs a specific job. Usually this means that it encapsulates a function.

Todo:Create a decorator to easily encapsulate a function.

from distlog import to
val = 10

with to('print int', arg=val):
    print(val)

Parameters:

• msg (string) – defines the goal of this subtask
• args (list) – parameters for the goal
• kwargs (dict) – context for log messages of this task

Return type:

Task

class distlog.JSONFormatter(fmt=None, datefmt=None)

Formatter to convert to JSON format.

encoding

Describe the encoding used.

Return string:encoding indicator

format(record)

Format to JSON.

Use the logging Formatter to convert the LogRecord data to JSON for network transport.

Parameters:record – LogRecord instance Returns:JSON encoded record contents.

class distlog.PickleFormatter(fmt=None, datefmt=None)

Formatter to convert to pickle format.

encoding

Describe the encoding used.

Return string:encoding indicator

format(record)

Format to pickle.

Use the logging Formatter to convert the LogRecord data to pickle format or network transport.

Parameters:record – LogRecord instance Returns:pickled record contents.

class distlog.ZmqHandler(endpoint, context=None, system='P')

0MQ transport implementation.

emit(record)

Do whatever it takes to actually log the specified logging record.

setFormatter(fmt)

Set the formatter for this handler.

set_topic(encoding)

Set message topic elements.

Creates the topic strings for log and performance messages. :param system: the system topic :param encoding: the encoding topic

class distlog.Task(_id, msg, *args, **kwargs)

Define the scope and context for other tasks and log messages.

A Task defines the context for other tasks and log messages. It is a context manager that is ususally created by calling one of the functions task() or to() where task() creates a new top-level context and to() creates a subtask.

The only real difference between the two is the way the id parameter is defined. A task() creates a UUID as id and to() creates a sequence number as id. import_task() is an alternative for to(), to be used when the subtask resides in another process.

When the __enter__() method of the Task is called it will emit a log message at INFO level signalling the begin of the scope. Thereafter all log messages will belong to the same scope until a subtask is created with the to() function or until the with section terminates.

When the __exit__() method of the Task is called another log message at the INFO level signals the completion of the scope. The cause of leaving the with section, with an exception or normally, controls which log message is produced.

Parameters:

• _id (string) – Either absolute (task) or relative (to) task id.
• msg (string) – Task description, may contain % formatting.
• args (list) – Parameters for the msg parameter.
• kwargs (dict) – Context definition. You may provide as many named parameters as needed. They are stored as key/value pairs in the context and are added to the LogRecord when it is created.

bind(**kwargs)

Add key/value pairs to the context.

Add the key/value pairs in kwargs to the dataset that is eventually returned by the context property.

Parameters:kwargs (dict) – dict with key/value pairs

context

Assemble the context.

The assembled context is added to the LogRecord instance as the context attribute. The assembly consists of the key identifying this log entry with the data provided when creating the context and any data added to it by the bind() function.

Vartype:dict containing the context data.

get_foreign_task()

Determine task id for a foreign task.

The returned ID can be passed over the network in an undefined manner and used on the other side as the first parameter for the function import_task(). Doing so will link both tasks (although in separate processes) toghether in one related set of log messages.

Return string:child task identification string.

get_next_task()

Identify the new subtask.

Every subtask spawned from this scope is identified by the current scopes identifier suffixed by the subtask sequence number. This method increments that sequence number to create a new scope.

Return int:subtask sequence number.

id

Property produces the context’s unique id.

parent

Parent id if available.

Provides the value of the parents id property or None if there is no parent.

Vartype:string Id of the parent component.

success(msg, *args)

Set success report message.

When the context manager is terminated successfully and success() has been called on the context then the msg is used to report the status in a log message.

Parameters:

• msg (string) – The log message to display
• args (list) – optional format parameters for the msg

class distlog.LogContext

Log message context.

The log message context is a stack of Task entries. Tasks form the context for individual log messages. Tasks can be constructed from subtasks which is why a stack structure is required.

The LogContext itself is a class with only a single instance which acts as a singleton and is defined as a module global in this file.

pop()

Remove top element of the stack.

Pop and return the topmost element of the stack.

Return type:Task the removed element.

push(action)

Push element onto the stack.

Parameters:action (Task) – item to add to the stack.

top

Produce element on top of stack.

Vartype:Task the element on top of the stack.

class distlog.LogRecord(name, level, pathname, lineno, msg, args, exc_info, func=None)

LogRecord replacement.

This class replaces the standard LogRecord class. It provides a single change: when an instance is created the instance is extended with a context attribute containing the current scope’s context.