Imports and the module system

import looks like a declaration and is actually a function call that runs code. Most import bugs — circular imports, “works from this directory but not that one”, a module-level side effect firing twice in tests — come from not knowing what that call does. This note is the mechanism; the syntax variants are a footnote to it.

What import x does

import sys

# 1. cache hit?  every import goes through sys.modules first
if "x" in sys.modules:
    x = sys.modules["x"]
else:
    # 2. find it: walk sys.meta_path finders, which search sys.path
    # 3. create an empty module object and register it in sys.modules
    # 4. execute x.py top to bottom, filling the module's namespace
    ...
# 5. bind the name `x` in the importing scope

Three consequences fall out of this sequence:

• A module executes once per process. The second import x anywhere returns the cached object, so module-level code — a connection pool, a registry, logging.basicConfig — runs exactly once. That is the reason module globals behave like singletons.
• The empty module is registered before it runs. During step 4 the module exists in sys.modules half-built, which is exactly what a circular import trips over. Circular imports.
• from x import y imports all of x. It runs the whole module and then binds one name. It is not cheaper, and it copies the binding: rebinding x.y later does not update your y. This is why patch("x.y") has to target where the name is looked up, not where it is defined — Fixtures and Patch in pytest.

Where it looks: sys.path

sys.path is searched in order: the script’s directory (or '' for the current directory in a REPL), then PYTHONPATH, then the stdlib and site-packages. The first entry is the source of the classic “works when I run it from here” bug:

python src/app/main.py       # sys.path[0] = src/app  -> `import app` fails
python -m app.main           # sys.path[0] = cwd      -> package imports work

python -m runs a module as part of its package, which is why it is the right way to run anything that uses package-relative imports, and why the src/ layout plus an editable install exists — see pyproject.toml and Building Packages. Manipulating sys.path at runtime works and is a smell; it means the project is not installed.

Packages, __init__.py and __all__

A directory with __init__.py is a regular package; the file runs when the package is imported and is the place to curate the public surface:

# app/__init__.py - re-export the API, hide the layout.
from .config import Settings
from .client import Client

__all__ = ["Settings", "Client"]     # what `from app import *` exposes

__all__ only governs * imports and what documentation tools consider public; it does not stop from app.client import _internal. A directory without __init__.py is a namespace package — importable, spread across multiple sys.path entries, no init code. Fine for plugin trees, a surprise everywhere else; write the empty file.

Absolute versus relative: from app.config import Settings versus from .config import Settings. Absolute is unambiguous and survives being copied elsewhere; relative survives renaming the top-level package. Inside a package either is fine — the failure is mixing them, and using relative imports in a file you also run as a script (ImportError: attempted relative import with no known parent package, because a script has no package).

Import-time cost and lazy imports

Every top-level import runs at startup, so a CLI that imports pandas to print --help pays 400 ms for nothing. The tools:

def export_csv(rows):
    import pandas as pd            # function-local: pay only when called
    return pd.DataFrame(rows).to_csv()

import importlib
mod = importlib.import_module(f"plugins.{name}")   # dynamic, by string

importlib.import_module is the public API for importing by name; __import__ is the hook it wraps and is not meant to be called directly. Function-local imports are also the cheap fix for an optional dependency and a workaround for a circular import — the second use is a signal to refactor, not a solution. python -X importtime -c "import app" prints the tree of what your startup actually costs.

importlib.reload(x) re-executes a module in place but does not update objects that already hold references to the old classes. It is a REPL tool, not something production code should rely on.

Where it breaks in production

• Side effects at import. A module that connects to a database or reads config at the top level runs that code the moment anything imports it — including the test collector. Put it in a function.
• Two copies of one module. Running python app/main.py and importing app.main elsewhere gives sys.modules both __main__ and app.main, two separate module objects; class identity checks and singletons break.
• Shadowing. A local json.py or test.py wins over the stdlib because the script directory is first on sys.path. The error is usually AttributeError: module has no attribute on something that obviously has.
• from x import * in application code. Names appear from nowhere, linters cannot see them, and a later addition to x silently overrides a local. Acceptable in an __init__.py that sets __all__; nowhere else.
• Import inside a hot loop. The sys.modules lookup is cheap but not free; hoist it if it is in the inner loop of a tight function.

Related

• Circular imports - the partially-initialised module, with the four fixes.
• pyproject.toml and Building Packages - src/ layout, editable installs, why python -m.
• Fixtures and Patch in pytest - patch where the name is looked up.
• ES Modules - the JavaScript contrast: live bindings, static analysis.