User guide¶
Configuration¶
A basic way to start is to just use it as you would use the default Python logger.
import uvlog
logger = uvlog.get_logger()
logger.set_level('DEBUG')
logger.debug('debug message')
By default the library provides a pre-configured root logger with a standard stderr handler and a text formatter, so it can provide reasonable log output out of the box.
For more sophisticated configuration you can use uvlog.configure()
function
which is similar to the Python dictConfig (however the format is slightly different).
root_logger = uvlog.configure({
'loggers': {
'': {
'levelname': 'ERROR',
'handlers': ['log.txt']
}
},
'handlers': {
'./log.txt': {
'class': 'QueueStreamHandler',
'formatter': 'json'
}
}
})
The configuration extends the standard BASIC_CONFIG
configuration by default, which has
default ‘stdout’, ‘stderr’ handlers and ‘text’, ‘json’ formatters preconfigured as well as the root logger.
If you want to do the entire configuration from scratch, use uvlog.clear()
first and then
uvlog.configure()
.
{
"loggers": {
"": { # logger name, "" for the root logger
"level": "INFO", # log level
"handlers": ["stderr"] # list of handler names (destinations) for 'handlers' dict
}
},
"handlers": {
'stderr': { # stderr and stdout are reserved for these special types of outputs
"class": "StreamHandler", # handler class
"level": "DEBUG", # log level for this particular handler
"formatter": "text", # assigned formatter key from 'formatters' dict
}
},
"formatters": {
"text": { # you can use any names here, however the two default formatters are 'text' and 'json'
"type": "TextFormatter",
"timespec": _default_timespec,
"format": _default_format,
},
"json": {"type": "JSONFormatter"},
},
}
Logging¶
You can pass extra parameters directly to the log record.
logger.info('my name is {name}', name='John')
Errors can be passed as in the standard logging module.
logger.error('something happened', exc_info=ValueError())
Child loggers can be created using get_child()
method.
app_logger = uvlog.get_logger('app')
service_logger = logger.get_child('service')
service_logger = uvlog.get_logger('app.service') # alternative way
Note that a logger is weak by default. It means that it eventually will be garbage collected if there are no live references to it. To create a persistent logger you need to pass persistent=True flag there.
Why do you want a persistent logger? Mostly because you want to have some logger-specific settings without need to pass a reference along your code. However in this case you should probably consider adding it to the configuration dict. All loggers created by the :py:func:`uvlog.configure` function are persistent.
app_logger = uvlog.get_logger('app', persistent=True)
Context variables¶
You can either provide your own context variable or use the default uvlog.LOG_CONTEXT
to provide
useful context for your log records.
logger = uvlog.get_logger()
async def handle_request(request):
uvlog.LOG_CONTEXT.set({'request_id': request.headers['Request-Id']})
logger.info('new request received')
await call_system_method()
return Response('OK')
async def call_system_method():
logger.info('calling a system method')
In the example both ‘new request received’ and ‘calling a system method’ messages will have the same request_id value assigned to their context.
2024-04-01T17:04:37 | INFO | | new request received | None | {'request_id': '123'}
2024-04-01T17:04:37 | INFO | | calling a system method | None | {'request_id': '123'}
Now you can cat logs.txt | grep 123 to output the whole request log chain,
or use the JSONFormatter
for the output,
aggregate your logs and send them to your ELK for example.
Sampling¶
Sampling allows to decrease amount of logs in a system by randomly picking and handling only some of them. There are many sampling mechanisms and you are encouraged to use them in your log aggregation services.
However, this library provides a simple sampling for loggers. sample_rate
defines a rate at which logs will be sampled by this logger
(i.e. probability for a log record to reach the log handlers).
Values above 1 are ignored and sampling mechanism is considered to be disabled for them.
To enable sampling globally just pass sample_rate
in your log config as follows.
root_logger = uvlog.configure({
'loggers': {
'': {
'sample_rate': 0.25
}
}
})
Sampling rules
Sampling mechanism is disabled for log messages of ‘WARNING’ and above.
If Context variables are used and the log context is not empty, then the first logger in the chain will determine whether the entire chain will be sampled. The reason behind this is to sample the whole chain for each incoming request (API call, task execution, etc.).
To illustrate this, imagine you have an incoming request in your server request handler, where you also set a context variable. Now, if sampling is enabled, the first log call ‘new request received’ will determine by a dice roll whether to sample the whole request chain. So you will either see all log messages for the whole request chain or nothing at all.
root_logger = uvlog.configure({
'loggers': {
'': {
'sample_rate': 0.25
}
}
})
system_logger = root_logger.get_child('system')
async def handle_request(request):
uvlog.LOG_CONTEXT.set({'request_id': request.headers['Request-Id']})
logger.info('new request received') # may be sampled here
await call_system_method()
logger.info('request finished') # 'new request received' determines whether it will be sampled
return Response('OK')
async def call_system_method():
system_logger.info('calling a system method') # 'new request received' determines whether it will be sampled
If Context variables are used, and ‘WARNING’ or ‘ERROR’ log record is encountered, the whole request chain after this record will be sampled to provide useful information for error handling. On the example below the error message forces subsequent logs in the chain to be sampled.
async def handle_request(request):
uvlog.LOG_CONTEXT.set({'request_id': request.headers['Request-Id']})
logger.info('new request received') # may be sampled here
await call_system_method()
logger.info('request finished') # 'something happened!' will force this record to be sampled
return Response('OK')
async def call_system_method():
system_logger.error('something happened!') # will be sampled
If you want to disable these rules, you can set sample_propagate
to False
for the root logger. This will disable all the log chaining mechanisms, and each logger becomes independent.
root_logger = uvlog.configure({
'loggers': {
'': {
'sample_rate': 0.25
'sample_propagate': False
}
}
})
Optimization¶
In a concurrent environment use the QueueStreamHandler
If you have a log aggregation stack such as ELK consider using the JSONFormatter
instead of the text formatter
Consider providing a more efficient JSON serializer for the JSONFormatter
such as orjson.
uvlog.JSONFormatter.serializer = orjson.dumps
Finally, use logging levels wisely and do not log too much in the production environment, or at least use Sampling.