The logging Module for Structured Logs

🏷️ Modules and Imports / Built-in Modules for Engineers

📚 All-in-One📖 Theory Only💻 Code Only

When you start building scripts that run for hours or handle critical tasks, simple print() statements quickly become a problem. You lose track of what happened, when it happened, and whether something went wrong. The logging module solves this by giving you a way to record events with timestamps, severity levels, and consistent formatting — all without cluttering your terminal output.

⚙️ What is Structured Logging?

Structured logging means your log entries follow a consistent pattern. Instead of seeing random text like "user logged in", you get a clean record such as "2025-04-01 14:30:00 — INFO — User 'admin' logged in from IP 192.168.1.10". This makes it easy to search, filter, and analyze logs later.

The logging module is built into Python, so you don't need to install anything extra. It gives you: - Five standard severity levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Timestamps automatically added to each entry - Flexible output — send logs to console, files, or both - Custom formatting to match your team's standards

🛠️ Basic Setup for a Simple Script

To start using logging, you configure a logger and set a minimum level. Any message below that level is ignored.

Import the module: import logging
Create a logger: logger = logging.getLogger(name)
Set the minimum level: logger.setLevel(logging.INFO)
Add a handler to send output somewhere: handler = logging.StreamHandler()
Attach the handler: logger.addHandler(handler)

Once configured, you log messages using: - logger.debug("Detailed info for troubleshooting") - logger.info("Normal operation message") - logger.warning("Something unexpected but not critical") - logger.error("A failure occurred") - logger.critical("System cannot continue")

📊 Comparison: print() vs logging

Feature	print()	logging
Timestamps	Manual	Automatic
Severity levels	None	Five built-in levels
Output control	Terminal only	Console, files, network
Filtering	Not possible	Filter by level or content
Production ready	No	Yes

🕵️ Logging to a File

For scripts that run unattended, writing logs to a file is essential. You replace the StreamHandler with a FileHandler.

Create a file handler: handler = logging.FileHandler("app.log")
Set its level: handler.setLevel(logging.INFO)
Define a format: formatter = logging.Formatter('%(asctime)s — %(levelname)s — %(message)s')
Apply the format: handler.setFormatter(formatter)
Add to logger: logger.addHandler(handler)

Now every log entry gets written to app.log with a timestamp and level name.

🧩 Structured Logging with JSON Format

Modern systems often expect logs in JSON format because it's easy for log aggregators (like Elasticsearch or Splunk) to parse. You can create a custom formatter that outputs JSON.

A simple approach is to define a formatter class that extends logging.Formatter and overrides the format() method to return a JSON string containing: - timestamp: the current time - level: the severity level name - message: the log message - module: the script or module name - function: the function where the log was called

This gives you structured, machine-readable logs without extra dependencies.

🧪 Common Pitfalls for Beginners

Forgetting to set the level: If you don't call setLevel(), the default is WARNING, so DEBUG and INFO messages will be silently ignored.
Creating multiple handlers: Each handler duplicates output. If you see duplicate log lines, check if you're adding the same handler twice.
Using print() inside functions: Once you switch to logging, avoid mixing print() and logger calls — it creates inconsistent output.
Not using name: Using getLogger(name) ties the logger to your module name, which helps when your project grows.

📝 Putting It All Together

A typical structured logging setup for a script looks like this:

Import the logging module
Create a logger with getLogger(name)
Set the logger level to INFO
Create a FileHandler pointing to a log file
Define a JSON-friendly formatter
Add the handler to the logger
Use logger.info(), logger.error(), etc. throughout your code

When you run the script, the log file will contain clean, timestamped entries. You can tail the file with tail -f app.log to watch events in real time, or grep for specific levels like ERROR to find problems quickly.

🧠 Key Takeaways

The logging module replaces print() for serious scripts
Use levels to control how much detail you see
File handlers let you persist logs for later review
Structured formats like JSON make logs machine-readable
Always set a level — otherwise DEBUG and INFO messages disappear

Start with a simple console logger, then add file output as your script grows. Once you're comfortable, experiment with JSON formatting to prepare for production environments.

The logging module provides a standardized way to record runtime events from your Python code, outputting them as structured messages with timestamps, severity levels, and context.

🛠 Example 1: Basic log message to console

This shows the simplest way to send a single log message to the terminal.

import logging
logging.warning("This is a warning message")

📤 Output: WARNING:root:This is a warning message

🛠 Example 2: Different severity levels

This demonstrates the five standard log levels and which ones appear by default.

import logging
logging.debug("Debug message — detailed diagnostic info")
logging.info("Info message — general operational info")
logging.warning("Warning message — something unexpected")
logging.error("Error message — a problem occurred")
logging.critical("Critical message — serious failure")

📤 Output: WARNING:root:Warning message — something unexpected
📤 Output: ERROR:root:Error message — a problem occurred
📤 Output: CRITICAL:root:Critical message — serious failure

🛠 Example 3: Configure logging level and format

This shows how to set a minimum level and customize the output format with timestamp and level name.

import logging
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s | %(levelname)-8s | %(message)s"
)
logging.info("Engineer started data processing")
logging.debug("Reading sensor values from device")
logging.error("Sensor timeout — retrying")

🛠 Example 4: Log to a file instead of console

This shows how to redirect all log messages into a persistent file for later review.

import logging
logging.basicConfig(
    filename="engine_run.log",
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s"
)
logging.info("Engine started")
logging.warning("Coolant temperature above threshold")
logging.error("Motor stall detected — shutting down")

📤 Output: (no console output — all messages written to engine_run.log)

🛠 Example 5: Structured log with extra context data

This shows how to attach additional key-value data to a log message for better debugging.

import logging
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s | extra=%(extra)s"
)
logger = logging.getLogger("sensor_monitor")
logger.info("Pressure reading", extra={"extra": {"sensor_id": "PT-101", "value": 2.45}})
logger.error("Pressure out of range", extra={"extra": {"sensor_id": "PT-101", "value": 8.12, "threshold": 5.0}})

📤 Output: 2025-03-20 14:30:15,126 | INFO | Pressure reading | extra={'sensor_id': 'PT-101', 'value': 2.45}
📤 Output: 2025-03-20 14:30:15,127 | ERROR | Pressure out of range | extra={'sensor_id': 'PT-101', 'value': 8.12, 'threshold': 5.0}

Comparison Table

Feature	`print()`	`logging` module
Severity levels	None	DEBUG, INFO, WARNING, ERROR, CRITICAL
Timestamp	Manual	Automatic via format string
Output destination	Console only	Console, file, network, etc.
Structured data	Manual string formatting	Extra dict parameter
Production ready	No	Yes — designed for engineers