Skip to content
System nominalUTC --:--:--Articles 041
MODULE 05 Lesson 15 / 15 40 MIN

Testing and static analysis

PHPStan Levels and a CI Pipeline

What PHPStan catches and what it cannot, how the levels build up, freezing existing errors in a baseline, and analysing only the diff in CI.

PHPStan reads your code and reports the things it can prove are wrong before anything runs. It found a real bug in the code for this lesson in under a second, and the same run stayed silent about a method that returned the wrong branch. Knowing which of those two outcomes to expect is most of what it takes to use the tool well.

The reason to care is that a test suite only covers the paths you thought of. A static analyser covers every path, including the ones you did not write a test for, and it covers them without executing anything. This lesson runs PHPStan 2.3 on a real class, reads the error, fixes it, and then wires the tool into CI in a way that survives a codebase with thousands of existing errors.

PHPStan proves some things and cannot prove others

PHPStan builds a model of what every expression's type could be, propagates that through assignments, calls and branches, and reports the places where the code does something the types do not allow. Passing a string to a function declared to take an integer is a proof. Returning a float from a method declared to return int is a proof. Neither of those examples needs a test to demonstrate.

What it cannot do is catch a wrong idea that type-checks. A discount rule that takes ten percent off when the business wanted fifteen has no type error in it. A method that subtracts the wrong value from the right type of value is invisible. Static analysis removes a large class of bugs and leaves the semantics to your tests, which is why the two tools sit next to each other rather than replacing each other.

The practical consequence is that a clean analyser run is not the same claim as a passing test suite. A clean run says every operation in the program is valid for the types involved. It does not say the program does what anyone wanted.

The levels are a ladder you climb one rung at a time

PHPStan ships eleven levels, 0 through 10, and they are cumulative: level 5 runs everything levels 0 to 4 check as well. The current documentation summarises the rungs like this.

LevelWhat it adds
0Basic checks, unknown classes and functions, unknown methods called on $this, wrong argument counts, definitely undefined variables.
1Possibly undefined variables, unknown magic methods and properties on classes with __call or __get.
2Unknown methods on all expressions rather than only $this, and validation of PHPDoc types.
3Return types and the types assigned to properties.
4Basic dead code: always-false instanceof, dead else branches, unreachable code after a return.
5Types of arguments passed to methods and functions.
6Missing type hints.
7Partially wrong union types: calling a method that exists on only some members of a union.
8Calling methods and accessing properties on nullable types.
9Strict about explicit mixed: the only thing you may do with it is pass it to another mixed.
10Strict about implicit mixed as well, so a missing type is reported too. Added in PHPStan 2.0.

Level 10 is new in PHPStan 2.0, which is why older articles describe the ladder as ending at 9. If you have read that the maximum level is 9, the article predates the release. --level max is an alias for the highest level in whatever version you are running, so it climbs automatically when you upgrade.

Level 5 is where an existing codebase gets real value

Level 5 is the first rung that checks arguments, and argument checking is where the high-value bugs live: a string passed where an integer is expected, an array passed to something expecting a countable, a null that slipped through a guard. It is also the first rung that does not demand annotations you have not written yet, because arguments have native types in any codebase written against PHP 7 or later.

Here is a small class with one genuine defect, and the configuration that analyses it:

final class Discount
{
    /** @param array<string, int> $rules */
    public function __construct(private array $rules = [])
    {
    }

    public function amountFor(string $code, int $minorUnits): int
    {
        return floor($minorUnits * 0.1);
    }

    public function isKnown(string $code): bool
    {
        return array_key_exists($code, $this->rules);
    }
}

PHPStan 2.3 at level 5 reports this, with the error identifier printed underneath the message so that you can suppress or baseline the specific rule rather than the file:

$ vendor/bin/phpstan analyse --no-progress
Note: Using configuration file /srv/app/phpstan.neon.

 ------ --------------------------------------------------------------------- 
  Line   Discount.php
 ------ --------------------------------------------------------------------- 
  16     Method Shop\Money\Discount::amountFor() should return int but returns
         float.
         🪪  return.type
 ------ ---------------------------------------------------------------------


 [ERROR] Found 1 error

floor() returns a float, and the method promises an int. Under PHP's coercion rules the call site would have received a float that happens to be whole, which is exactly the kind of value that passes casual testing and then fails in array_sum or a strict comparison. The fix is to make the conversion explicit:

public function amountFor(string $code, int $minorUnits): int
{
    return intdiv($minorUnits * 10, 100);
}

Run the same analysis at level 0 and it reports nothing. That is not a bug in the tool. Level 0 does not check return types, so the defect is outside what it looks at, and this is the single most useful thing to understand about the ladder: a green run at a low level tells you the code is not broken in the ways that level examines.

The -l flag is ignored when a config file sets a level

If phpstan.neon contains a level: key, passing -l 0 on the command line does not change it. PHPStan loads the configuration, which supplies the level, and the flag loses. To compare levels, either edit the config or point at a different one with -c. This trips up almost everyone the first time they try to measure what a higher level would find.

A baseline freezes today's errors so tomorrow's are visible

A codebase that has never run an analyser has errors, and demanding all of them be fixed before the tool is allowed in CI means the tool never gets adopted. The baseline records the current errors in a file and stops reporting them, so the analyser only speaks about code that changed.

vendor/bin/phpstan analyse --generate-baseline
parameters:
	ignoreErrors:
		-
			message: '#^Parameter \#1 \$string of function str_pad expects string, int given\.$#'
			identifier: argument.type
			count: 1
			path: src/Report.php

The entry records the message pattern, the error identifier, how many times it occurs, and which file. Include the file from the main configuration:

includes:
    - phpstan-baseline.neon

parameters:
    level: 5
    paths:
        - src
        - tests
    tmpDir: var/phpstan

The property that makes a baseline honest is that it is not a mute button. Fix an error that the baseline covers and the next run fails with an ignore.unmatched error telling you the pattern no longer matches anything:

 ------ ----------------------------------------------------------------------- 
  Line   Report.php
 ------ ----------------------------------------------------------------------- 
         Ignored error pattern #^Parameter \#1 \$string of function str_pad
         expects string, int given\.$# (argument.type) in path
         /srv/app/src/Report.php was not matched in reported errors.
         🪪  ignore.unmatched (non-ignorable)
 ------ ----------------------------------------------------------------------- 

Unmatched patterns are reported as errors by default, so the baseline shrinks or it fails. If you want a baseline that can be regenerated cheerfully without ever shrinking, set reportUnmatchedIgnoredErrors: false, and accept that the file will grow until nobody reads it.

The PHPStan documentation makes the trade-off plain: inline @phpstan-ignore annotations carry a line number and allow a comment explaining why, and a baseline entry carries neither. Use the baseline to get a legacy tree under analysis today, and use inline annotations with a reason for anything new.

Native types are the contract, docblocks are the detail

A native type declaration is enforced at runtime and read by every tool. A docblock is read only by tools, and it can say things the language cannot express. That division tells you where each belongs.

public function withOverride(string $pair, float $rate): self
{
    $next = $this->table;
    $next[$pair] = $rate;

    return new self($next);
}

The three parameters have native types, so this method needs no docblock at all. A docblock on it would be decoration, and decoration drifts. The constructor is a different case, because private array $table says the parameter is an array and says nothing about what is in it:

/**
 * @param array<string, float> $table
 */
public function __construct(private array $table)
{
}

PHPStan validates PHPDoc types from level 2 upward, which means a docblock that references a parameter which does not exist is itself an error. Writing one by hand on a method that has no such parameter produces this, and the identifier names the problem precisely:

  Line   Rates.php
 ------ -------------------------------------------------------- 
  19     PHPDoc tag @param references unknown parameter: $table
         🪪  parameter.notFound

The rule that follows from both examples: do not write a docblock that restates a native type, and do write one for the shapes a native type cannot describe. When a docblock and a native declaration disagree, the analyser reports an error rather than guessing, which is the behaviour you want from a tool that is supposed to be checking your claims.

Generics describe what is inside a collection

PHP has no generics in the language, and PHPStan understands them in docblocks. The syntax that pays for itself first is the array shape, because collections of arrays are everywhere:

final class MoneyBag
{
    /** @param array<string, int> $minorUnits */
    private function __construct(private array $minorUnits)
    {
    }

    public function add(string $currency, int $minorUnits): void
    {
        $this->minorUnits[$currency] = ($this->minorUnits[$currency] ?? 0) + $minorUnits;
    }

    /** @return array<string, int> */
    public function all(): array
    {
        return $this->minorUnits;
    }
}

With that annotation in place, every read of all() has an element type of int, and a call that treats it as anything else is reported at the call site:

  Line   Report.php
 ------ --------------------------------------------------------------------- 
  13     Parameter #1 $string of function str_pad expects string, int given.
         🪪  argument.type

The fix is to convert at the boundary, which is what the type was telling you all along:

$rows[] = sprintf('%s %s', $currency, str_pad((string) $minorUnits, 5, '0', STR_PAD_LEFT));

Template tags go further, and the useful one is on a method that returns whatever a callback returns rather than a fixed type:

/**
 * @template T
 * @param callable(Product): T $map
 * @return list<T>
 */
public function map(callable $map): array
{
    return array_map($map, $this->products);
}

Now a method that declares @return list<string> and passes a callback returning int is a reported error, and the message names both types. That check caught a real mistake in the code for this lesson:

  Line   ProductCollection.php
 ------ ----------------------------------------------------------------------- 
  39     Method Shop\Catalog\ProductCollection::skus() should return list<stri
         ng> but returns list<int>.
         🪪  return.type

A class-level @implements IteratorAggregate<int, Product> does the same for iteration, so a foreach over the collection knows the element type without a docblock on the loop variable. Two details about generics in docblocks are worth knowing before you use them. The type must resolve in the file's namespace, so IteratorAggregate written without a leading backslash and without a use statement resolves to Shop\Catalog\IteratorAggregate and produces a generics.wrongParent error. And these tags are only read by tools, so a wrong one misleads your editor and the analyser together while the program keeps running.

Analyse the diff, not the tree

A full-tree analysis that takes four minutes is a check people route around. The useful long-term shape is to keep the whole-tree run for a nightly job and make the pull request check report only on files the branch changed.

#!/usr/bin/env bash
# Analyse only the PHP files this branch changes, not the whole tree.
set -euo pipefail

base="${1:-origin/main}"

files=$(git diff --name-only --diff-filter=ACMR "$base"...HEAD -- '*.php' | while read -r file; do
    [ -f "$file" ] && printf '%s\n' "$file"
done)

if [ -z "$files" ]; then
    echo "No changed PHP files against $base."
    exit 0
fi

echo "Analysing changed files against $base:"
printf '  %s\n' $files

vendor/bin/phpstan analyse --no-progress $files

Run against a branch that changed one file, it prints the file and then the error, and it exits 1 so the pipeline fails:

$ ./bin/analyse-changed.sh base
Analysing changed files against base:
  src/ProductCollection.php
Note: Using configuration file /srv/app/phpstan.neon.

 ------ ----------------------------------------------------------------------- 
  Line   ProductCollection.php
 ------ ----------------------------------------------------------------------- 
  39     Method Shop\Catalog\ProductCollection::skus() should return list<stri
         ng> but returns list<int>.
         🪪  return.type
 ------ ----------------------------------------------------------------------- 

The three dots in the diff are the part people get wrong. git diff base...HEAD compares HEAD against the merge base of the two branches, so it lists the files your branch changed and only those. Two dots compare the two tips directly, and once the base branch has moved on, a two-dot diff also lists every file that the base branch changed after your branch started. Those files are not yours, and failing the check on them is the fastest way to get an analyser job switched off. The --diff-filter=ACMR keeps added, copied, modified and renamed files and drops deletions, because a deleted path cannot be passed to the analyser. The [ -f "$file" ] guard catches the rename case where the old name is gone.

Three further settings make the tool behave in CI the way it does on your machine. tmpDir gives the result cache somewhere writable, which is what turns a repeat run from seconds into something negligible. parallel spreads the analysis over available cores. And pinning the version with the caret in composer.json while committing composer.lock means the analyser in CI is the analyser you ran locally, so a new release cannot fail your build on a Tuesday.

Once the diff job is green and stable, raise the level by one in the config, generate a fresh baseline, and let the next few pull requests fix what it found. That is the whole adoption strategy, and it works because the baseline turns an unbounded task into a bounded one.

Lab 30 minutes

Take a class you have already written and run PHPStan at level 0 and then level 5, fixing nothing in between. Record which errors appear at 5 and not at 0.

Fix one of them properly, regenerate the baseline, and then fix a second error that the baseline covers, so you see the ignore.unmatched failure for yourself.

Then add the diff script and confirm it exits 0 on a branch with no PHP changes and 1 on a branch with one broken file. Write down which exit code your CI configuration would have interpreted as success.

Show the solution and the reasoning

The expected pattern is that level 0 comes back green and level 5 reports return types and argument types. If level 0 already reports errors, they are almost always unknown classes from a missing autoloader or misspelled method names, and those are worth fixing before anything else because every later level depends on the analyser being able to resolve your classes at all. That is why the project needs the autoload section in composer.json and a composer dump-autoload before the first run.

The second part is the one that teaches the baseline's behaviour. Fixing an error that the baseline covers does not silently succeed. You get:

Ignored error pattern #^Parameter \#1 \$string of function str_pad expects string, int given\.$#
(argument.type) in path /srv/app/src/Report.php was not matched in reported errors.
🪪  ignore.unmatched (non-ignorable)

The fix is to delete that entry from phpstan-baseline.neon, which is normally a one-line edit, or to regenerate the whole file. Deleting the entry is better when the file is shared, because regenerating it can add or remove unrelated entries and produce a diff nobody can review.

On the exit codes, PHPStan returns 1 when it finds errors and 0 when it does not. The failure mode to avoid is a pipeline step that pipes the command into head or tee without set -o pipefail, because the exit status of a pipeline is the status of its last command, and phpstan ... | head exits 0 every time. The script above ends with the analyser as the last command and runs under set -euo pipefail, so the status survives. If your CI shows a green analyser step on a branch you know is broken, that pipe is the first thing to check.

One judgement call is worth naming. Analysing only the diff means new code is held to the level while old code is not, and that asymmetry is deliberate. The nightly full-tree run is what stops it becoming permanent, and the baseline is what keeps that run from failing on day one.

Progress

Mark this lesson done and it stays marked in this browser.

Course complete

That is the last lesson of Modern PHP, from Syntax to Services. If the material was useful, the practice is what makes it stick: rebuild the final project from an empty directory without looking back at the lesson.