Configuration and logging

Configuration changes behavior between environments without changing source code. Logging records what the program did. Treat both as public operational interfaces: define schemas, precedence, redaction, and failure behavior deliberately.

Separate configuration from secrets

Use versioned TOML for non-secret defaults and environment variables for deployment-specific values and secrets.

# config.toml
[service]
host = "127.0.0.1"
port = 9000
request_timeout_seconds = 10.0

[logging]
level = "INFO"

Python 3.11 and later include tomllib for reading TOML:

from pathlib import Path
import tomllib


def read_toml(path: Path) -> dict[str, object]:
    with path.open("rb") as file:
        return tomllib.load(file)

tomllib does not write TOML. Add a dependency only if writing is truly required.

Validate into typed settings

Environment values are always strings and require parsing.

from dataclasses import dataclass
import os


@dataclass(frozen=True)
class Settings:
    host: str
    port: int
    request_timeout_seconds: float
    api_token: str


def load_settings() -> Settings:
    token = os.environ.get("APP_API_TOKEN")
    if not token:
        raise RuntimeError("APP_API_TOKEN is required")

    try:
        port = int(os.getenv("APP_PORT", "9000"))
        timeout = float(os.getenv("APP_REQUEST_TIMEOUT_SECONDS", "10"))
    except ValueError as error:
        raise RuntimeError("Port and timeout must be numeric") from error

    if not 1 <= port <= 65_535:
        raise RuntimeError("APP_PORT must be between 1 and 65535")
    if timeout <= 0:
        raise RuntimeError("Timeout must be positive")

    return Settings(
        host=os.getenv("APP_HOST", "127.0.0.1"),
        port=port,
        request_timeout_seconds=timeout,
        api_token=token,
    )

Load settings once at the composition root, then pass the resulting object to components. Avoid reading environment variables throughout domain code. Document precedence, for example: command-line flags override environment values, which override TOML, which overrides defaults.

Configure logging once

Library modules should obtain a named logger and avoid configuring global handlers:

import logging

logger = logging.getLogger(__name__)


def import_records(count: int, source: str) -> None:
    logger.info("Import started", extra={"record_count": count, "source": source})

The application entry point owns configuration:

import logging


def configure_logging(level: str) -> None:
    numeric_level = getattr(logging, level.upper(), None)
    if not isinstance(numeric_level, int):
        raise ValueError(f"Invalid log level: {level}")

    logging.basicConfig(
        level=numeric_level,
        format="%(asctime)s %(levelname)s %(name)s %(message)s",
    )

Use parameterized logging rather than eager string formatting:

logger.debug("Loaded %d records from %s", count, source)

Formatting is deferred when the log level is disabled.

Choose levels consistently

• DEBUG: diagnostic detail useful during investigation;
• INFO: meaningful lifecycle or business milestones;
• WARNING: unexpected condition handled without aborting current work;
• ERROR: operation failed and requires attention;
• CRITICAL: process or system can no longer continue safely.

Log exceptions at the boundary that decides how failure is handled:

try:
    synchronize()
except ConnectionError:
    logger.exception("Synchronization failed", extra={"operation": "synchronize"})
    raise

logger.exception() includes the active traceback. Avoid logging the same exception at every layer.

Protect sensitive data

Never log passwords, tokens, session IDs, authorization headers, or complete personal records. Prefer allowlisted fields over trying to redact arbitrary objects. Correlation IDs should be opaque and should not encode private information.

Pitfalls

• committing .env files containing secrets;
• silently accepting malformed values and falling back to defaults;
• ambiguous precedence between files, environment, and flags;
• configuring handlers inside reusable libraries;
• logging full request or configuration objects;
• catching an exception only to log and continue in an invalid state;
• unbounded high-volume logs in a loop.

Checklist

• Every setting has a type, default policy, and validation rule.
• Required secrets fail fast and are never committed or logged.
• Configuration precedence is documented and tested.
• Settings are loaded at the composition root.
• Logging is configured once by the application.
• Events include useful context without sensitive values.
• Exceptions are logged once at the handling boundary.

Practice

Build a loader with TOML defaults, environment overrides, and one CLI override. Test missing secrets, malformed numbers, and precedence. Add logs for startup and one failed operation, then inspect the output as if you were diagnosing the failure without source access.