pytest — tests as a habit

100% coverage, bug still shipping

Here is a small pricing function with one bug: it forgets the surcharge that applies when a parcel is both international and express. Every line is reachable, every branch can go both ways, and the bug lives in none of them — it lives in a combination. Three suites, each one a reasonable thing a careful person would write:

This is the honest reading of a coverage report: it tells you what your tests executed, never what they checked. A suite can execute every line while asserting almost nothing — and even 100% branch coverage only promises each if went both ways at some point, not that the interesting combinations were ever tried. Coverage is a floor, not a target: use it to find the code nobody tests at all, and don't mistake a green percentage for a tested program.

The fix here costs one decorator, because pytest makes combinations cheap:

import pytest

@pytest.mark.parametrize("intl", [False, True])
@pytest.mark.parametrize("express", [False, True])
@pytest.mark.parametrize("weight", [2.0, 25.0])
def test_price(intl, express, weight):
    assert price(weight, intl, express) == expected(weight, intl, express)

Stacked parametrize decorators multiply, so those three lines are eight test cases with eight separate names in the report — and when one fails, pytest tells you which combination, not just "test_price failed". That is the practical difference from writing a loop inside one test.

The suite that passes on your machine

The second thing to get right early is fixtures, which is pytest's dependency injection: a function decorated with @pytest.fixture is requested by name in a test's parameter list. The trap is its scope. Below are four tests sharing one module-scoped list, run in every possible order:

The dangerous part is the top row. pytest runs tests in declaration order by default, so this suite is green on your machine, every time — and stays green right up until someone runs it under pytest-xdist (which distributes tests across processes), or pytest-randomly, or reruns a single failure with pytest path::test_name. Then it fails, in a way that looks like a flaky product bug rather than a test-design bug.

The cure is not simply narrowing the scope: notice that the second option had to rewrite each test to establish its own preconditions. A test that only passes after another test ran is not an independent test — it is one long test written in four pieces. Default to scope="function" (which is pytest's default) and widen it only for genuinely expensive, genuinely read-only setup: a database container, a downloaded model, a parsed corpus.

import pytest

@pytest.fixture                          # scope="function" — fresh for every test
def cart():
    return []

@pytest.fixture(scope="session")         # built once for the whole run — read-only things only
def model():
    return load_expensive_model()

def test_add_one(cart):                  # requested by NAME, injected by pytest
    cart.append("x")
    assert len(cart) == 1

Two more things you get for free. Assert rewriting: pytest rewrites the bytecode of your assert statements so a failure prints the actual values on both sides — which is why you never need assertEqual. And pytest.raises is how you assert that something fails:

def test_rejects_negative_weight():
    with pytest.raises(ValueError, match="positive"):
        price(-1, intl=False, express=False)

Fixtures, parametrize, monkeypatch — the three that do the work

Above, scope was a hazard. Here it is a dial, and the thing worth seeing is a plain count: how many times the fixture body actually runs. A fixture is built once per unit of its scope — once per test at scope="function", once per file at "module", once for the whole run at "session" — and every test that asks for it inside that unit gets the same object. The panel runs the same three tests both ways and shows the instances that existed, the report pytest printed, and which tests shared what.

Two more fixtures ship with pytest and you will use them constantly. tmp_path hands each test its own empty directory (a pathlib.Path), which is how you test file-writing code without a finally block that deletes things. monkeypatch sets an attribute, an environment variable or a dict entry for the duration of one test and puts it back afterwards — no teardown code, and no chance of forgetting it on the error path. And @parametrize, which you met above as a way to multiply combinations, has a second job here: it names them, so a report line points at one case.

The fixture that is built once is not wrong — a parsed corpus, a loaded model, a started container all belong at "session". What makes the second option fail is that the shared object is mutable and mutated: the first test writes a file into the directory every later test then inspects. The rule that survives contact with a real suite is widen the scope, freeze the object. If a session fixture must hand out something writable, have it hand out a factory (tmp_path_factory is exactly that) so each test still gets its own.

import time
import pytest

@pytest.fixture(scope="session")             # built once for the whole run
def corpus(tmp_path_factory):
    d = tmp_path_factory.mktemp("corpus")    # a NEW directory each call — the factory pattern
    (d / "a.txt").write_text("hello")
    yield d                                  # everything after the yield is teardown
    print("torn down once, at the end of the session")

def test_token_is_stamped_with_a_fixed_clock(monkeypatch):
    monkeypatch.setattr(time, "time", lambda: 1_700_000_000.0)
    assert issued_at() == 1_700_000_000       # put back automatically when the test ends

Note the yield: a fixture that yields runs its teardown afterwards, at the end of its scope's unit — which is the other half of what scope decides, and the half that bites when a session fixture leaves a container running. monkeypatch is the same idea with the bookkeeping already written: it records every change you make through it and reverses them in the reverse order, whether the test passed, failed or raised. Setting time.time by hand and restoring it in a finally is the same thing with two more places to get it wrong.

Next in Day 9: packaging & mypy --strict, then exercise py-08, which is the first suite here whose fixtures build a real wheel.

Second opinion (taught here — these corroborate): pytest docs · pytest fixtures · coverage.py.