How to monitor an application (logging)
The key to developing a healthy application is to set up monitoring correctly. The alpha and omega of this process for us is logging.
One of Python's built-in libraries (logging) is designed to provide a simple way to record these messages (logs), which we can later use as a data source when analyzing the application.
import logging
logger = logging.getLogger(__name__)
def drink_coffee(user, coffee_count):
logger.debug(f'User {user.id} just drank another cup of coffee')
return coffee_count + 1An elementary example of logging looks like this. Using the code, we can log messages about the process and the current state during active use of the application.
Why log?
- A properly written log makes it easy to trace the error and the processes that led to it
- it gives us the possibility to make an analysis of the use of the application and its different parts
Main parts of the logging process
- logger - exposes an interface that we can then use in code
- formatter - specifies the message format and the necessary parameters for logging the message
- handler - "sends" messages (logs) to the right destination - saves them to a file, sends them to the console, etc.
- filter - allows more control over the selection of logged messages to be displayed
Each part is important in the logging and log analysis process and it is desirable to learn what each part is responsible for.
Hierarchy of loggers
The loggers in the python library "logging" are arranged in a hierarchy according to the name used to create them.
import logging
console_output = logging.StreamHandler()
root_logger = logging.getLogger()
# root_logger.addHandler(console_output)
# root_logger.setLevel(logging.DEBUG)
# příklad přidávání handleru na logger (zachycování zpráv loggeru)
x_logger = logging.getLogger('x')
x_y_logger = logging.getLogger('x.y')
x_z_logger = logging.getLogger('x.z')
root_logger.info('zpráva 1')
x_logger.info('zpráva 2')
x_y_logger.info('zpráva 3')
x_z_logger.info('zpráva 4')Hierarchie loggerů z této ukázky vypadá následovně:
If we capture messages from the root logger, we get in the output:
message 1
message 2
message 3
message 4
But the output from the X logger will only be:
message 2
message 3
message 4
Logger X is below the main(root) logger in the hierarchy and therefore does not capture what is sent directly to the main logger.
We only capture message 3 from logger Y and message 4 from logger Z because they are at the same level and have the same "parent".
Record severity levels
Each message in the log can have different "severity" levels, allowing us to filter the logs according to what is needed at the time.
We have five levels (sometimes six counts):
DEBUG (+ TRACE)
INFO
WARNING
ERROR
FATAL
Each level has its own meaning and it is up to the developer to correctly determine the severity of the record.
Basic logging rules
- Always use the logging library of the language - (for us python - logging lib, js - dev console, node.js - Winston), never write your own library
- Log at the right level - makes it very easy to work with data
- Log to multiple categories/loggers - depending on the context/part of the application
- Write useful messages without unnecessary information
- Use English - avoids problems with file encoding (standard EN is ASCII only) and language barrier problems
Why is logging important?
Keeping a log of your application's operation can be beneficial for many reasons, the main ones being:
- control of data movement on the application - sometimes required by state/EU laws
- diagnostics of application performance
- statistics on the usage of different parts