Backend / Python core / Stdlib / 06_pathlib_logging.md

pathlib and logging

Updated 5 interview angles 4 min read source
On this page3
  1. pathlib — modern paths
  2. logging — structured logging done right
  3. Interview angle

pathlib and logging

Two stdlib modules everyone uses but few use well.

pathlib — modern paths

Path replaces os.path string manipulation with object-oriented file operations. Cross-platform, more readable, and harder to misuse.

python 
from pathlib import Path

p = Path("/var/log/app.log")
p.name           # 'app.log'
p.stem           # 'app'
p.suffix         # '.log'
p.parent         # PosixPath('/var/log')
p.parts          # ('/', 'var', 'log', 'app.log')
p.is_absolute()  # True

Building paths

python 
home = Path.home()
config = home / ".config" / "myapp" / "settings.json"   # uses /
str(config)   # '/home/user/.config/myapp/settings.json'

The / operator joins paths and handles separators correctly on each OS.

File I/O

python 
p = Path("notes.txt")

p.read_text()                    # whole file as str
p.read_bytes()                   # whole file as bytes
p.write_text("new content")
p.write_bytes(b"binary")

with p.open("r") as f:
    for line in f:
        ...

Inspection

python 
p.exists()
p.is_file()
p.is_dir()
p.is_symlink()
p.stat()        # os.stat result
p.stat().st_size

Globbing

python 
project = Path("/home/me/project")

list(project.glob("*.py"))           # top-level .py files
list(project.rglob("*.py"))          # recursive
list(project.glob("**/test_*.py"))   # recursive (alt syntax)

Mutating

python 
p = Path("data.txt")
p.touch()              # create empty if missing
p.unlink(missing_ok=True)   # delete (3.8+ has missing_ok)
p.rename("newname.txt")
# like rename but overwrites if dest exists
p.replace("newname.txt")

# equiv to `mkdir -p`
Path("dir").mkdir(parents=True, exist_ok=True)

Common pattern: temporary work

python 
import tempfile
from pathlib import Path

with tempfile.TemporaryDirectory() as tmp:
    workdir = Path(tmp)
    (workdir / "input.txt").write_text("...")
    # cleanup automatic on exit

logging — structured logging done right

print for debugging is fine in scripts. For real applications, use logging. It supports levels, hierarchical loggers, configurable handlers, and structured output.

Levels

text 
DEBUG   — verbose; for development
INFO    — normal operations
WARNING — something unusual but not failing
ERROR   — operation failed
CRITICAL — service-level failure

Higher levels include lower ones — setting INFO shows INFO, WARNING, ERROR, CRITICAL but suppresses DEBUG.

Module-level usage

python 
import logging

# named logger per module
logger = logging.getLogger(__name__)

logger.debug("about to fetch %s", url)
logger.info("user logged in: %s", user_id)
logger.warning("retry %d for %s", n, url)
logger.error("failed to save: %s", err)
# includes traceback automatically
logger.exception("crashed during X")

logger.exception captures sys.exc_info() and logs at ERROR level with traceback. Always use it in except blocks.

Use lazy formatting

python 
# correct — format string applied only if level enabled
logger.debug("user %s did %s", user, action)

# wrong — string built even when DEBUG is disabled
logger.debug(f"user {user} did {action}")

The %-style with deferred args is a small but real performance win, and integrates with structured logging tools.

Configuring handlers

python 
import logging

# Set the root logger
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    handlers=[
        logging.StreamHandler(),                          # stderr
        logging.FileHandler("app.log"),                   # file
    ],
)

For larger apps, use logging.config.dictConfig to load configuration from a dict (or yaml/toml file).

Logger hierarchy

Logger names with dots form a hierarchy: myapp.api.auth is a child of myapp.api, which is a child of myapp, which is a child of root. Configuration propagates down the tree.

python 
logging.getLogger("myapp").setLevel(logging.DEBUG)
# all of myapp.* now log at DEBUG

This is why logger = logging.getLogger(__name__) is the standard idiom — __name__ becomes the dotted module path, integrating with any hierarchy config.

Structured / JSON logging

For aggregators (Datadog, ELK, CloudWatch Insights), emit JSON:

python 
import logging
import json

class JsonFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "ts": record.created,
            "level": record.levelname,
            "logger": record.name,
            "msg": record.getMessage(),
            "extra": getattr(record, "extra", {}),
        })

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logging.getLogger().addHandler(handler)

Or use python-json-logger, structlog, or loguru for less boilerplate.

Adding context: extra and LoggerAdapter

python 
logger.info("payment received", extra={"user_id": 42, "amount": 100})

For per-request context (request_id, user_id), use LoggerAdapter or contextvars to flow context through async tasks.

Interview angle 5

  • “Why pathlib over os.path?” - cross-platform separators, methods that chain, a real Path type instead of a string, and explicit semantics. Most of the modern stdlib accepts Path directly, so the interop cost that used to justify os.path is gone.
  • “What logging level should production run at?” - INFO normally, DEBUG only while investigating an incident. The level is a runtime decision, which is the argument for configuring it from the environment rather than in code.
  • “Why getLogger(__name__) rather than logging.info(...)?” - it gives you the module hierarchy, so a handler, filter or level can be set for one package without touching the rest. Calling the module-level functions configures the root logger and takes that away.
  • “How do you do request-scoped logging in async code?” - contextvars for the request id, surfaced through a LoggerAdapter or a filter. Thread-locals do not work here, because many coroutines share a thread.
  • “Why lazy formatting?” - logger.debug("x %s", value) only formats when the level is enabled; an f-string builds the message every time regardless. On a hot path at INFO, that is real work thrown away.
Previous
dataclasses — boilerplate-free data classes