Python Logging: Debugging, Monitoring, and Security with the Logger Class

When to Use the Logger Class?

1. Debugging and Troubleshooting: Detailed and customizable logs simplify identifying and rectifying issues in complex applications.
2. Monitoring and Analysis: Logging is essential for monitoring behavior, identifying performance bottlenecks, and analyzing usage patterns in production environments.
3. Security and Compliance: Comprehensive logging is vital for auditing and security analysis in applications handling sensitive information or needing to adhere to specific compliance requirements.

How to Use the Logger Class?

Save the following code in a file named Logger.py, and then open another Python file main.py in the same root directory and use from Logger import Logger to import the package.

Logger.py

import logging
from logging import handlers

class Logger(object):
    level_relations = {
        'debug': logging.DEBUG,
        'info': logging.INFO,
        'warning': logging.WARNING,
        'error': logging.ERROR,
        'crit': logging.CRITICAL
    }  # relationship mapping

    def __init__(self, filename, level='info', when='D', backCount=3,
                 fmt='%(asctime)s - %(pathname)s[line:%(lineno)d] - %(levelname)s: %(message)s'):
        self.logger = logging.getLogger(filename)
        format_str = logging.Formatter(fmt)  # Setting the log format
        self.logger.setLevel(self.level_relations.get(level))  # Setting the log level
        console_handler = logging.StreamHandler()  # on-screen output
        console_handler .setFormatter(format_str)  # Setting the format
        th = handlers.TimedRotatingFileHandler(filename=filename, when=when, backupCount=backCount,encoding='utf-8')  # automatically generates the file at specified intervals
        th.setFormatter(format_str)  # Setting the format
        self.logger.addHandler(sh)  # Add the object to the logger
        self.logger.addHandler(th)

This code defines a custom Logger class that can be used to create and configure loggers in Python.

1. The Logger class is initialized with several parameters:
   • filename: The name of the log file where the logs will be saved.
   • level: The logging level, which determines the severity of the messages to be logged (e.g., 'debug', 'info', 'warning', 'error', 'crit').
   • when: The interval at which the log files should be rotated (e.g., 'S' for seconds, 'M' for minutes, 'H' for hours, 'D' for days, 'W' for weeks).
   • backCount: The number of backup log files to retain.
   • fmt: The format of the log messages, which includes the time, file path, line number, log level, and the log message itself.
2. Inside the __init__ method, the logger is configured as follows:
   • The logging level is set based on the provided level.
   • A format string is created for formatting the log messages.
   • A console handler is created for logging messages to the console.
   • A rotating file handler is created for logging messages to the specified log file.
3. Finally, the logger is set up to use both the console handler and the rotating file handler for logging messages.

main.py

from Logger import Logger

log = Logger('log\\all.log', level='debug')
log_error = Logger('log\\all.log', level='debug')

log.logger.info("This is an info message)")
log_error.info("This is an error message")

This Python code shows how to use the Logger package to write log information to a file. It first initializes two Logger objects, one for general logging and one for error logging. It then uses these objects to write log messages at different levels, such as info and error. The log messages are written to the specified log file, allowing for easy monitoring and troubleshooting of the application.

Customizing Parameters

The Logger class provides several parameters that can be customized based on the specific logging needs. These parameters include:

1. filename: The name of the log file where the logs will be written.
2. level: The log level, which determines the severity of the logs (e.g., debug, info, warning, error, crit).
3. when: The frequency at which the log files should be rotated (e.g., 'D' for daily, 'H' for hourly).
4. backCount: The number of backup log files to retain after rotation.
5. fmt: The format of the log messages, allowing customization of the timestamp, log level, file name, line number, and message content.

By adjusting these parameters, the Logger class can be tailored to meet the specific logging requirements of different projects and applications.

Example 1: Customizing Log Level and Format

log = Logger('custom.log', level='warning', fmt='%(asctime)s - %(levelname)s - %(message)s')
log.logger.debug("This is a debug message - won't be logged")
log.logger.warning("This is a warning message - will be logged")

Example 2: Specifying Log Rotation Parameters

log = Logger('rotation.log', when='midnight', backCount=7)
log.logger.info("Logging with rotation based on time")

Conclusion

Logging is crucial in software development for monitoring, troubleshooting, analyzing, and securing applications. Python's logging class offers a convenient and customizable way to implement logging with various configuration options. This class enables developers to tailor logging setups to meet project needs, ensuring comprehensive and efficient logging for debugging, monitoring, and compliance purposes. The Logger class empowers developers to harness the power of logging for robust application development and maintenance.

Happy logging!

Thank you for taking the time to explore data-related insights with me. I appreciate your engagement. If you find this information helpful, I invite you to follow me or connect with me on LinkedIn. Happy exploring!👋

Also published here.