Automated Testing for WordPress Plugins

The export screen works on the developer's machine. It is tested by hand once, with a note whose title is Test, and the CSV opens correctly. A fortnight later a customer with an apostrophe in their company name gets a file where the name has a backslash in it, and the count in the header is one higher than the number of rows. Nothing is broken in the sense of throwing. The plugin did exactly what it was told to do with the data it never expected.

A test suite is the tool for that class of bug, and it is worth being honest about the size of the investment. WordPress ships a test suite that boots the real application inside PHPUnit, with a database, the real hook system and the real REST server. Getting it running for a plugin takes an afternoon the first time. Once it runs, the data you were never going to type by hand becomes a fixture you write once and reuse.

The test suite boots WordPress, then wraps it in PHPUnit

The WordPress test library is not a mock and not a simulator. It is a set of PHP files that require the real wp-settings.php with a test configuration, which means the code under test runs against the real database, the real WP_Query, the real hooks and the real REST server. What the library adds on top is isolation and helpers.

Almost every test class extends WP_UnitTestCase, which extends WP_UnitTestCase_Base in tests/phpunit/includes/abstract-testcase.php. That base class does four things a plain PHPUnit\Framework\TestCase does not:

// tests/phpunit/includes/abstract-testcase.php, WP_UnitTestCase_Base::set_up()
$this->start_transaction();          // BEGIN, with autocommit off
$this->expectDeprecated();           // fail on _deprecated_* calls
add_filter( 'wp_die_handler', array( $this, 'get_wp_die_handler' ) );

A transaction is opened before each test and rolled back after it. That is why a test can create twenty posts, change options and switch the current user without any cleanup code: the writes never survive the ROLLBACK at the top of tear_down(). It also means a test that wants to commit something has to opt out of the transaction, which is a decision to make deliberately.

The second difference is what happens to wp_die(). The base class replaces the die handler with one that throws WPDieException, so a handler that calls wp_die() on a failed capability check does not end the test run, it fails one test with a readable message. The third is the factory, covered below. The fourth is that deprecated calls fail the test rather than logging a notice, which is how a plugin finds out that an API it uses was deprecated in the version it claims to support.

Two ways to get the environment

The library has to be installed somewhere the bootstrap can find it, and there are two established routes.

The scripted route is install-wp-tests.sh, distributed with WP-CLI's scaffold command and documented in the WordPress handbook. It takes the database name, user, password and host, plus an optional WordPress version, and it downloads the test library and a WordPress checkout into $TMPDIR, then writes wp-tests-config.php. The version argument matters for a course like this one: passing 7.0.7 makes it fetch the matching tag of the wordpress-develop repository rather than trunk. The plugin then points at the result with the WP_TESTS_DIR environment variable, which is the convention the generated bootstrap uses.

The containerised route is wp-env, the WordPress development environment built on Docker. It starts a WordPress site and a separate tests site from a .wp-env.json, mounts your plugin into both, and can run a command inside the tests container, which is where PHPUnit lives. The advantage is that nobody has to install MySQL or match a PHP version by hand. It is the wrong choice for a shared CI runner that has no Docker, and it hides the location of the library, which makes the $WP_TESTS_DIR question disappear rather than answer itself.

Either way the plugin needs a bootstrap and a PHPUnit configuration.

// tests/bootstrap.php
$_tests_dir = getenv( 'WP_TESTS_DIR' );

if ( ! $_tests_dir ) {
    $_tests_dir = rtrim( sys_get_temp_dir(), '/\\' ) . '/wordpress-tests-lib';
}

require_once $_tests_dir . '/includes/functions.php';

/**
 * Load the plugin under test before WordPress finishes booting.
 *
 * @return void
 */
function wpsifero_notes_manually_load_plugin(): void {
    require dirname( __DIR__ ) . '/wpsifero-notes.php';
}
tests_add_filter( 'muplugins_loaded', 'wpsifero_notes_manually_load_plugin' );

require $_tests_dir . '/includes/bootstrap.php';

The muplugins_loaded hook is the important line. WordPress is loaded but no plugin has been loaded yet, so the plugin is included at the same point a must-use plugin would be, and every hook it registers is registered before any test runs. The alternative, activating the plugin through the database, works too and is slower.

Factories beat fixtures for WordPress data

Creating a post by hand in a test means writing eight array keys to get a row that WP_Query will return, and getting one of them wrong produces a test that passes for the wrong reason. The factories exist so that a test says what matters about its fixture and nothing else.

public function test_notes_are_listed_in_creation_order(): void {
    $author = self::factory()->user->create( array( 'role' => 'editor' ) );

    $first = self::factory()->post->create( array(
        'post_type'   => 'post',
        'post_status' => 'publish',
        'post_author' => $author,
        'post_title'  => 'First note',
    ) );

    $second = self::factory()->post->create( array( 'post_title' => 'Second note' ) );

    $this->assertNotSame( $first, $second );
}

self::factory() is a static method on the base class and returns the shared factory object, with a property per object type: post, user, term, comment, attachment, blog and more. Each create() takes an array of overrides and fills in defaults that produce a valid object. The overrides are where the test's meaning lives: the role of a user, the status of a post, the meta value the code under test reads.

Where a fixture is genuinely shared across a class, build it once in wpSetUpBeforeClass() rather than in every test. It runs before the class's tests, and wpTearDownAfterClass() is its counterpart. The base class commits the transaction after the class setup so the shared rows survive, which is why a fixture built there is visible to every test in the class and removed afterwards.

Test a hook by firing it and asserting the effect

The tempting assertion is the cheap one: that add_action() was called with the right arguments. It tells you nothing. A callback registered on the right hook can still write the wrong value, bail on the wrong condition or return nothing where the caller expects a string.

The assertion that pays is the effect. Fire the action with do_action(), then assert what changed in the database, what was echoed, or what the handler returned. The plugin's save handler is a good case, because it ends with wp_safe_redirect() and exit, and a handler that exits takes the PHPUnit process with it. A test intercepts the redirect and turns it into an exception, which both stops the exit and captures the URL.

public function set_up(): void {
    parent::set_up();

    $this->redirect = null;
    add_filter(
        'wp_redirect',
        function ( $location, $status ) {
            $this->redirect = array( $location, $status );

            throw new WPSifero_Notes_Redirect_Exception( (string) $location );
        },
        10,
        2
    );
}

public function test_handling_the_action_writes_the_note(): void {
    wp_set_current_user( self::factory()->user->create( array( 'role' => 'administrator' ) ) );

    $_GET  = array();
    $_POST = array(
        '_wpnonce'       => wp_create_nonce( 'wpsifero_notes_save' ),
        'wpsifero_title' => "O'Brien & Sons",
    );
    // What wp-settings.php does to a real request, including $_REQUEST.
    wp_magic_quotes();

    try {
        do_action( 'admin_post_wpsifero_notes_save' );
        $this->fail( 'The handler did not redirect.' );
    } catch ( WPSifero_Notes_Redirect_Exception $e ) {
        // Expected: the redirect is where the handler ends.
    }

    $notes = get_option( 'wpsifero_notes' );

    $this->assertSame( "O'Brien & Sons", $notes['obrien-sons']['title'] );
    $this->assertSame( admin_url( 'tools.php?page=wpsifero-notes' ), $this->redirect[0] );
}

Three details in that test are worth copying. wp_magic_quotes() reproduces what WordPress does to the superglobals at startup, including rebuilding $_REQUEST, so the test exercises the same slashing the real request carries and a missing wp_unslash() fails here. wp_create_nonce() has to run after wp_set_current_user() or the nonce is generated for the wrong user and the handler dies before writing anything. And tear_down() resets the current user to zero, so a test that forgets to log in gets a handler that fails the capability check rather than a false pass.

The negative case is a test too, and it is the one that keeps a security fix from being reverted. A subscriber with a valid nonce is stopped by the capability check, and because the test suite turns wp_die() into an exception, the assertion is expectException( 'WPDieException' ) followed by the same do_action() call.

Global state is what breaks the test that runs after yours

The transaction handles the database. It does not handle anything WordPress keeps in a global or in a static property, and a test suite where one test silently changes the behaviour of the next is worse than no suite.

When a test does need setup of its own, the inherited lifecycle is where it goes. set_up() and tear_down() are the per-test pair, and in the WordPress test library they are the snake_case methods, which is why a test class written in camelCase quietly overrides the wrong thing. Call parent::set_up() first and parent::tear_down() last; the parent is what opens the transaction and resets the globals.

A real passing run

The plugin for this module has fifteen tests over three files. They were run against WordPress 7.0.7 with PHP 8.5.6, MySQL 9.6.0 and the matching wordpress-develop 7.0.7 test library, from a fresh test database:

$ php /tmp/wpplugin-m5-lab/phpunit-9.6.phar
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 9.6.38 by Sebastian Bergmann and contributors.

...............                                                   15 / 15 (100%)

Time: 00:00.180, Memory: 61.47 MB

OK (15 tests, 44 assertions)

The Installing... line appears only on the first run, when the test library creates the database tables. The skipped groups are core test groups that are excluded unless asked for; none of them are plugin tests. A failing assertion prints the expected and actual values next to the file and line, which for a comparison test is usually enough to see the bug:

1) WPSifero_Notes_Demo_Failure_Test::test_sanitised_title_keeps_the_apostrophe
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'OBrien'
+'O'Brien'

/private/tmp/.../plugin/wpsifero-notes/tests/test-demo-failure.php:9

Some parts of a plugin are not worth a test

A test suite that tries to cover everything becomes a second codebase and stops being maintained. The parts of a plugin that earn a test are the ones where a wrong answer is silent and costly.

Worth testing: the sanitise and lookup paths, because the failures are invisible without adversarial data. Capability and nonce gates on every state-changing entry point, because the regression is a security hole. Anything that walks a request into the database or back out, including the export and import code that produced the opening bug. A hook that transforms data, tested by calling the filter and asserting the return value. The translation loading of a plugin that ships its own .mo files, because a wrong text domain produces no visible error.

Not worth much: that add_action() was called, since the effect test covers it. That a settings page renders its HTML, because the assertion ends up being a long string comparison that breaks every time somebody changes a label. That WordPress itself works, which is core's test suite's job and not yours. And pixel-level output of an admin screen, which a browser tool does better than PHPUnit ever will.

The honest limit of the whole approach is that the test suite boots WordPress synchronously. It does not exercise HTTP requests, JavaScript, the block editor, or anything that depends on a browser, and it does not catch a plugin being slow. Tests that need a real HTTP round trip go through the REST server object inside the same process, which is close but not the same as a request over the network. For anything beyond that, the suite is a fast correctness net and not a replacement for trying the plugin by hand before a release.

class WPSifero_Notes_Export_Test extends WP_UnitTestCase {

    public function test_export_keeps_titles_and_counts_rows(): void {
        $titles = array( "O'Brien & Sons", 'A "quoted" name' );

        foreach ( $titles as $title ) {
            self::factory()->post->create(
                array(
                    'post_type'   => 'post',
                    'post_status' => 'publish',
                    'post_title'  => $title,
                )
            );
        }

        $csv  = wpsifero_notes_export_csv();
        $rows = array_filter( explode( "\n", trim( $csv ) ) );

        // One header row plus one row per note.
        $this->assertCount( count( $titles ) + 1, $rows );

        foreach ( $titles as $title ) {
            $this->assertStringContainsString( $title, $csv );
        }

        $this->assertStringNotContainsString( "\\'", $csv );
    }
}