Testing in Python

The framework question is settled — pytest, for new code, without much argument. What is worth having an opinion about is what each test layer is for and where the suite stops paying for itself.

The layers, and what each catches

The ratio matters more than the definitions: many unit, some integration, few end-to-end. Inverting it produces a suite that takes twenty minutes, fails intermittently, and gets disabled. See Test Strategy and the Test Pyramid.

The other kinds — performance, security, acceptance — are not pyramid layers. They answer different questions and run on a different cadence.

unittest versus pytest

Both discover tests, both run in CI. The difference is how much ceremony sits between you and the assertion.

import unittest

class TestMath(unittest.TestCase):
    def setUp(self):
        self.a, self.b = 10, 5

    def test_addition(self):
        self.assertEqual(self.a + self.b, 15)

The same thing in pytest — a function, a plain assert, and setup expressed as an argument:

import pytest

@pytest.fixture
def pair():
    return 10, 5

def test_addition(pair):
    a, b = pair
    assert a + b == 15

The fixture system is the real difference, not the syntax. setUp runs for every test in the class whether that test needs it or not; a fixture runs because a test asked for it by name, and fixtures compose into each other.

Rule of thumb: use unittest when you cannot add a dependency, or the codebase is already built on it. Use pytest otherwise. pytest runs unittest.TestCase classes unchanged, so migration is incremental rather than a rewrite.

The pytest features worth knowing by name

Parametrize, which turns one test into many cases with real names:

@pytest.mark.parametrize(
    "value,expected",
    [(0, "zero"), (1, "one"), (-1, "negative")],
)
def test_describe(value, expected):
    assert describe(value) == expected

A failure reports test_describe[-1-negative], so you know which case broke without reading the body.

Assertion rewriting is why the bare assert is enough. pytest rewrites the expression at import so a failure shows both sides:

E       assert {'a': 1, 'b': 3} == {'a': 1, 'b': 2}
E         Differing items:
E         {'b': 3} != {'b': 2}

Fixture scope is the performance lever people miss. A database container built per test is the usual reason a suite is slow:

@pytest.fixture(scope="session")
def db():
    with Postgres() as p:      # once for the whole run
        yield p

Widen the scope and you must keep tests from leaking state into each other — usually a transaction rolled back per test on top of a session-scoped connection.

What makes a test worth keeping

• One behaviour per test, named so the failure line explains itself.
• Assert on behaviour, not implementation. A test that breaks on every refactor is measuring the wrong thing.
• Deterministic. No real clock, no network, no ordering dependency between tests.
• Fast enough to run on save. A suite you run once a day catches things a day late.

Gotcha: 100% coverage is not the goal. Coverage records which lines executed, not which behaviours were verified — a test with no assertion at all still covers its lines. Use it to find untested areas, not as a target.

Related

• Fixtures, parametrize and scope — fixtures and markers in depth
• Mocking external APIs — the boundary you mock
• Test Strategy and the Test Pyramid — how much of each layer