Skip to content
System nominalUTC --:--:--Articles 041
MODULE 05 Lesson 15 / 18 35 MIN

Testing and typing

Type Annotations and a Checker in CI

Use type annotations with mypy or pyright, run the checker in CI, cope with packages that ship no stubs, and answer it with a branch rather than an ignore.

Type annotations do not make Python a statically typed language. The interpreter reads them, stores them, and otherwise ignores them, which means a mislabelled function runs happily and fails where the wrong value is used. What annotations buy you is a second pass over the code by a program that reasons about types, and that pass catches a class of bug before a test does.

The example below is real and it is the kind of bug tests miss. A lookup function returns either an item or nothing, the caller indexes the result without checking, and the test suite is green because the test only exercises the path where the item exists. The checker sees the missing branch on the first run.

An annotation is data the interpreter does not act on

Write a function with a wrong annotation and call it with the wrong type.

def double(value: int) -> int:
    return value * 2


print(double("seven"))
sevenseven

The function ran, multiplied a string, and printed a string. The annotation was a comment as far as CPython was concerned. If you want to read the annotations back at runtime, they are available as a dictionary:

import typing

print(typing.get_type_hints(double))
{'value': <class 'int'>, 'return': <class 'int'>}

That dictionary is what tools read. Nothing verifies it against the runtime values unless you install something that does, and this course does not, because runtime type checking slows the program down and duplicates what a checker and a test already do.

The typing essentials you will actually use

Modern annotations use built-in generic syntax: list[str], dict[str, float], tuple[int, ...]. Those forms work from Python 3.9 onward, and a module that needs them on an older interpreter gets them from from __future__ import annotations, which makes annotations strings and never evaluates them. What matters more than the syntax is choosing the right shape for the data.

FormUse it when
int | None A value may be absent. The old spelling is Optional[int], which means the same thing: a union with None, not an optional argument.
int | str A value may be one of several types. Union[int, str] is the pre-3.10 spelling.
Callable[[str], int] A function is passed in. The list is the parameter types, the second element is the return type.
Literal["short", "long"] A string or number has a closed set of allowed values. The checker rejects a typo.
TypedDict A dictionary with a fixed set of keys and per-key types, usually from JSON or a config file.
Protocol You depend on the shape of an object rather than a concrete class, and you do not want to change that class's inheritance.

The union operator is worth internalising because it reads as what it is. Optional[int] sounds like an argument with a default, and people write it on parameters that have no default. int | None says the value may be None, and if a caller ignores that, the checker tells them. Here is a small module that uses five of these forms together; the dictionary passed to the renderer is a TypedDict rather than dict[str, object], which is what lets the checker tell an int from a str in the values.

from collections.abc import Callable, Iterable
from typing import Literal, Protocol, TypedDict


class Row(TypedDict):
    sku: str
    quantity: int


class Sink(Protocol):
    def write(self, line: str) -> None: ...


Formatter = Callable[[Row], str]
Mode = Literal["short", "long"]


class Collector:
    def __init__(self) -> None:
        self.lines: list[str] = []

    def write(self, line: str) -> None:
        self.lines.append(line)


def render(rows: Iterable[Row], formatter: Formatter, mode: Mode) -> str:
    prefix = ">> " if mode == "long" else ""
    return "\n".join(prefix + formatter(row) for row in rows)

Collector does not inherit from Sink and never mentions it, and the checker accepts it anyway. Protocol describes a shape rather than a family: an object is a Sink if it has a write method taking a string and returning nothing. That is how you type a collaborator you do not control. The cost is that a typo in the method name shows up as an argument error at the call site.

Read the checker error, then fix the code

Install the checker into the same virtualenv as pytest. Either mypy or pyright will do; mypy is written in Python and installs with pip, so the rest of this lesson uses it. The runs below are mypy 2.4.0 on Python 3.14.7. Check both versions on your own machine, because checkers tighten their default rules between releases.

python -m pip install mypy
python -m mypy --strict inventory.py

Now the module that motivated the lesson. It looks up an item by SKU, and the lookup can legitimately find nothing:

from typing import TypedDict


class Item(TypedDict):
    sku: str
    quantity: int


def find(items: list[Item], sku: str) -> Item | None:
    for item in items:
        if item["sku"] == sku:
            return item
    return None


def stock_for(items: list[Item], sku: str) -> int:
    return find(items, sku)["quantity"]

The return type of find is honest; the body of stock_for ignores it. The checker's complaint names the operation and the type, and nothing else:

inventory.py:21: error: Value of type "Item | None" is not indexable  [index]
Found 1 error in 1 file (checked 1 source file)

Notice what the checker did not need: execution, a test, or input data where the SKU is absent. To get the same signal from a test you would have to think of the missing item, and the test suite in this project does not, because it only checks a SKU that exists:

1 passed in 0.00s

The fix is a branch, not a cast and not an ignore. A cast tells the checker to stop looking; an ignore tells it the same thing more loudly. A branch answers the question the type raised, and the answer is a decision about the API: what should happen when the item is absent?

def stock_for(items: list[Item], sku: str) -> int:
    item = find(items, sku)
    if item is None:
        raise KeyError(sku)
    return item["quantity"]
Success: no issues found in 1 source file

The checker found a hole in the logic, and closing it required a product decision. Raising KeyError is right when a missing SKU means the caller asked for something that should exist; returning 0 is right when it means no stock. Both are one line, and both beat the silent TypeError the original code produces at runtime.

What --strict turns on

--strict is a bundle of individual flags, including --disallow-untyped-defs and --warn-return-any. Turning it on in an existing codebase produces a wall of errors, most of them about functions you have not annotated yet. Leave it off until the annotations exist, then turn it on and keep it on, because strictness you switch off during a deadline is strictness you never get back. The mypy command-line documentation lists what the flag includes.

Packages without stubs are a normal condition

Most of the ecosystem is unannotated, and a checker that refused to work with it would be useless. The behaviour is per package and it depends on a marker file. A distribution that installs a py.typed marker is checked directly, annotations and all. One that does not, and for which no stub package exists, is a package the checker cannot read: it types the import as Any and everything derived from it, silently. Here is a real installed package with no stubs, imported by a file being checked strictly:

use_tab.py:1: error: Library stubs not installed for "tabulate"  [import-untyped]
use_tab.py:1: note: Hint: "python3 -m pip install types-tabulate"
use_tab.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
use_tab.py:3: note: Revealed type is "Any"
Found 1 error in 1 file (checked 1 source file)

The hint is the first thing to try, because typeshed publishes stubs for a long list of popular packages as separate distributions named types-*. The Revealed type is "Any" note is the part that matters even when the error is silenced, because Any is the value the checker uses for every result of that call. The good fix is a local stub file named tinytool.pyi, sitting next to the module it describes, declaring only the parts you use:

def run(spec: int) -> int: ...

The acceptable option is per-module configuration, which scopes the hole to the package that has it. In pyproject.toml that is a TOML table under the mypy section; in a mypy.ini it is a section named [mypy-yourpackage.*]. Both work, and the TOML form is what this course uses:

[[tool.mypy.overrides]]
module = ["tinytool.*"]
ignore_missing_imports = true

The option to avoid is ignore_missing_imports = true in the [tool.mypy] section itself. It silences the error for every package in the project, including the one that shipped its own stubs and would have caught a real mistake in your call. The difference between the two configurations is the difference between a known gap and an unknown one.

That silence is the reason to keep Any out of your own signatures and to treat an untyped dependency as a known hole. When you must have Any, write it explicitly and comment why, and consider a Protocol describing the two methods you actually call. A protocol you write is structural; Any is not.

The checker belongs in the same command as the tests

A type checker you run manually is a type checker you run on the day you write the annotations. Run it on every push, in the same job as pytest, and let it fail the job. Configure the settings in pyproject.toml so the local and CI commands are identical:

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q"

[tool.mypy]
python_version = "3.12"
strict = true
warn_unreachable = true
name: checks

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python -m pip install -r requirements.txt
      - run: python -m pytest
      - run: python -m mypy .

With that configuration in place, python -m mypy . reads pyproject.toml and needs no flags, and CI runs the same command you do. Two consequences are worth planning for.

First, the run gets slower as the project grows, and mypy caches results in .mypy_cache. Add that directory to .gitignore and let CI start cold rather than shipping a stale cache into the repository. Second, annotate new code as you write it rather than in a sweep later. A migration over a codebase of any size produces a week-long branch that conflicts with everything. Adding -> None to the test functions you write today costs seconds.

The thing a checker will not do is tell you whether your types are the right ones. It checks that your annotations agree with each other and with the operations you perform. A function that returns seconds and is annotated -> int passes, and so does one that returns milliseconds under the same annotation. That remains the test suite's job, which is why the two tools ship together and neither replaces the other.

Lab 35 minutes

Take a module you have already written in this course, or the top_words function from the previous lab, and run python -m mypy --strict on it. Fix every error by adding a parameter annotation, a return annotation, or a branch that handles the absent case. Do not add a # type: ignore comment anywhere.

Then add a function with a genuine hole in it: one that returns str | None and is called by another function that assumes a string. Run the checker, read the error, and fix it. Then write a test for the None path and confirm the test fails when you remove your fix.

Finally, add the checker to a CI configuration as a step that runs after pytest, and answer: which of the errors you fixed could a test have caught, and which one could only the checker have caught?

Show the solution and the reasoning

The annotated module. The annotations here are the minimum that satisfies --strict, which is one annotation on every parameter and one on the return, plus a type argument on every container:

import re
from collections import Counter

WORD = re.compile(r"[^\W\d_]+", re.UNICODE)


def top_words(text: str, limit: int = 3) -> list[tuple[str, int]]:
    if limit < 1:
        raise ValueError("limit must be at least 1")
    counts = Counter(WORD.findall(text.lower()))
    return counts.most_common(limit)

list[tuple[str, int]] is doing real work. Written as -> list, the elements are typed as Any, so the checker stops helping at exactly the point where you index into the result, and --strict reports the missing type argument separately. The annotation states that the caller gets pairs, which is the fact the tests depend on.

The optional-returning function and its caller:

def display_name(user_id: int) -> str | None:
    names = {1: "ana", 2: "bo"}
    return names.get(user_id)


def greet(user_id: int) -> str:
    return display_name(user_id)
solution.py:7: error: Incompatible return value type (got "str | None", expected "str")  [return-value]
Found 1 error in 1 file (checked 1 source file)

The error code in brackets is the part worth reading: [return-value] names the annotation on the return, not the value you returned. Adding a code to disable_error_code stops the checker looking at that category everywhere, which is almost never what you want.

The test for the None path is two lines, and it is the test that was missing from the suite in this lesson:

def greet(user_id: int) -> str:
    name = display_name(user_id)
    if name is None:
        return "Hello, guest"
    return f"Hello, {name}"


def test_greet_handles_an_unknown_user() -> None:
    assert greet(99) == "Hello, guest"

The branch is the fix, and it is an interface decision: an unknown user gets a greeting with no name in it. The test pins that decision. Replace the four lines of the fix with return f"Hello, {display_name(user_id)}" and the checker stays quiet, because an f-string accepts any type, while the test fails with assert 'Hello, None' == 'Hello, guest'. That pair of outcomes is the reason you want both tools: the checker guards the types, the test guards the words.

Notice the order of events. The checker found the hole with no test data, and the test then pinned the chosen behaviour. Neither step subsumes the other.

On the final question, a test could have caught the missing None branch if you had thought to write a test with an unknown id, and that is the cost of the test-only approach: the bug you fail to imagine is the bug you ship. The same asymmetry holds for the Item | None case in the lesson, where the existing suite exercised only a SKU that exists and would have stayed green through the crash. The reverse case is a function that satisfies every annotation and still returns the wrong number. Keep both tools, run both, and put the checker first because it is the cheap pass.

Progress

Mark this lesson done and it stays marked in this browser.