Cache Stampedes and Thundering Herds

An expiring cache key looks like a small event. One value becomes invalid, the next request rebuilds it, and the site carries on. That is true when requests arrive one at a time. It stops being true when a key is popular enough that dozens of requests are inside the server when it expires: they all miss together, they all build the same value, and the cache that was protecting the database has organised a simultaneous attack on it.

The third module showed you how to invalidate without lying. This lesson is about what the miss path costs when it stops being a single request, and it measures that cost instead of describing it. Everything below comes from a lab with Redis 8.10 and WordPress 7.0.7 on PHP 8.5.6, with 8 PHP worker processes and 40 or 64 concurrent clients, and every number is from a run you can repeat with the code in the lab.

A stampede is a race between a read and a write, and the read is not atomic with it

The standard read-through is three statements. Ask the cache for the key. If the answer is a miss, build the value. Store it. There is no point in that sequence where the key is reserved, so two requests that both reach the miss can both build. The window is exactly as long as the build takes, which is why the problem is invisible with fast values and severe with slow ones.

Here is that shape against the real thing. The endpoint does the same expensive work as the previous lesson, caches the result, and reports whether the request built it. Each run starts with flushall against Redis, so the cache is genuinely empty, and 32 clients are released at the same instant.

naive read-through, 32 concurrent requests, 3 runs
  responses 32, rebuilt 9,  p50 362 ms  max 389 ms
  responses 32, rebuilt 9,  p50 183 ms  max 249 ms
  responses 32, rebuilt 9,  p50 205 ms  max 252 ms

with wp_cache_add() as a mutex, 32 concurrent requests, 3 runs
  responses 32, rebuilt 1,  p50 144 ms  max 193 ms
  responses 32, rebuilt 1,  p50 147 ms  max 186 ms
  responses 32, rebuilt 1,  p50 159 ms  max 205 ms

Nine rebuilds from 32 requests, every time. The nine is not arbitrary: the server has eight PHP worker processes, so at most nine requests can be past the cache check before the first one finishes and writes the value. The size of your worker pool is therefore also the size of your worst-case duplicate work. More workers buy concurrency and pay for it in rebuilds.

The mutex version rebuilds once. The first request to miss wins wp_cache_add(), builds the value and publishes it; the other eight wait and then read it. Wall time for the burst drops from around 350 ms to around 150 ms, because the server stops running the same loop nine times over.

The mutex is wp_cache_add(), and its failure mode is its expiry

wp_cache_add() is the mutex because it is the only cache call that fails when the key exists. Read its contract in core rather than trusting the intuition: it returns true on success and false when the key and group already exist, and the persistent backends implement that as an atomic set-if-absent. Redis does it with a set that only succeeds on a missing key. Compare that with wp_cache_set(), which always writes, and with the read-through above, where the check and the write are two separate round trips with a build in between.

/**
 * Rebuild a cached value with at most one builder at a time.
 */
function read_expensive_value( $key, $lock_ttl = 5, $wait = 3.0 ) {
    $value = wp_cache_get( $key, 'myplugin' );

    if ( false !== $value ) {
        return $value;
    }

    // One process wins this; the rest are told the key already exists.
    if ( wp_cache_add( $key . '_lock', 1, 'myplugin', $lock_ttl ) ) {
        $value = expensive_build();
        wp_cache_set( $key, $value, 'myplugin', 300 );
        wp_cache_delete( $key . '_lock', 'myplugin' );
        return $value;
    }

    $deadline = microtime( true ) + $wait;
    do {
        usleep( 5000 );
        // The third argument forces a read from the persistent cache instead of
        // the copy this process already holds.
        $value = wp_cache_get( $key, 'myplugin', true );
    } while ( false === $value && microtime( true ) < $deadline );

    if ( false === $value ) {
        // Fail open: the holder died or is slower than the lock.
        $value = expensive_build();
        wp_cache_set( $key, $value, 'myplugin', 300 );
    }

    return $value;
}

Two arguments in that snippet are decisions, not boilerplate. The lock's expiry has to be longer than the slowest plausible rebuild, or the lock disappears while the holder is still working and every waiter becomes a builder. And the waiter has to give up eventually, or one process killed mid-build leaves every subsequent request spinning until its own timeout. The fail-open branch is the part people omit, and it is the difference between a cache problem and an outage.

What a dead holder costs was measured with the same 40 concurrent workers, a 1200 ms rebuild, and the lock expiry as the variable. With the expiry above the rebuild time, one process rebuilt. With the expiry below it, all forty did.

The runs above use a Redis client directly rather than WordPress, so the lock expiry can be varied and the rebuild can be a fixed 1200 ms. Each worker process records the moment it starts a rebuild; "concurrent" counts the rebuilds that began within one rebuild time of each other, and the spans confirm it: with the two-second expiry the forty workers produced one event, and with the 0.3 s expiry they produced forty events inside a 0.96 s window.

The 0.3 s row is the failure mode in one line. The lock expired 300 ms into a 1200 ms build, waiters timed out, and forty processes rebuilt the same value inside a one-second window. The wall time for the run doubled and the p95 rose while the cache was, by every metric a dashboard would show, working correctly. A lock with an expiry shorter than the work it protects does not prevent a stampede. It schedules one.

Probabilistic early expiry spreads rebuilds out, and it is the wrong tool for a burst

The second fix does not wait for the key to expire. Every reader of a key rolls a die that is biased by how close the key is to expiry and by how expensive the rebuild is, and a reader that wins the roll rebuilds early. Read the condition as now - (beta * ln(random())) >= expiry, which is one line in the application and needs no coordination between processes.

function read_with_early_refresh( $key, $ttl, $beta = 0.02 ) {
    $packed = wp_cache_get( $key, 'myplugin', true );

    if ( false === $packed ) {
        $value = expensive_build();
        wp_cache_set( $key, array( $value, microtime( true ) ), 'myplugin', $ttl );
        return $value;
    }

    list( $value, $written ) = $packed;
    $age = microtime( true ) - $written;

    // As the age approaches the TTL the chance of refreshing early rises.
    if ( ( $age - $beta * log( mt_rand() / mt_getrandmax() ) ) >= $ttl ) {
        $value = expensive_build();
        wp_cache_set( $key, array( $value, microtime( true ) ), 'myplugin', $ttl );
    }

    return $value;
}

Run that against 40 workers released at once and it does worse than the naive version: 40 rebuilds, all overlapping, a p95 of 636 ms. The technique is sound and the run is not a refutation of it. Every worker here reads a key written moments earlier and evaluates the same condition against the same age at the same instant. Early expiry decorrelates expiry times; it does nothing about a synchronised read.

Where it earns its keep is a site with many keys, uneven access, and traffic spread over time. Then it turns one visible cliff at each expiry into a scatter of early rebuilds, each paid for by a single unlucky request rather than by everybody. beta should be proportional to rebuild time divided by the cost of serving a hit. If you cannot estimate that ratio on the back of an envelope, the lock is easier to reason about, because its failure is visible and countable.

Serving stale data while refreshing hides the rebuild from every visitor

The third fix changes who waits. One process is elected to refresh the value in the background while every other request, including the elected one, serves the previous value. Requests stop being blocked for the duration of a build and the key never has a window where nothing is stored, at the price of allowing data that is one rebuild older than the TTL. For a popular post, a menu, a sitemap or a product list, that trade is almost always right. For a stock level or a balance, it is not, and no amount of engineering makes it right.

The measurement that matters for this technique is not the throughput, which is flat. It is the tail. Across 480 requests from 40 workers, the stale-serving variant recorded 23 rebuilds with 22 of them overlapping, and a p95 of 41 ms against 602 ms for the naive version. The rebuilds still happened, in about the same number. They were just invisible to the clients, because nobody was waiting for one.

Note what the shape of the storage has to become. A single key with a TTL cannot distinguish "fresh but expired" from "gone", so stale-while-revalidate stores the value with a long expiry and a separate freshness marker, and an elected worker refreshes when the marker is absent. That is a deliberate second key whose only job is to expire first. It is also where the mutex comes back: the election needs wp_cache_add(), or every worker that notices the stale marker will refresh at once, which is the same stampede with extra steps.

The transients API gives you none of this, and on a database-backed site it makes the miss worse

Transients get recommended for exactly this problem, and the recommendation is wrong for a mechanical reason: there is no add-if-absent on the transients API. There is get_transient() and set_transient(), and the second one always writes. A miss that ends in set_transient() therefore publishes unconditionally, so two processes that miss together both build and both overwrite, with the loser's work discarded. That is the naive read-through with extra options table traffic.

Measure it on the lab site and the transient endpoint is indistinguishable from the naive one: same 32-client burst, same nine rebuilds, same shape of distribution. The number is not the point. The point is where the writes go when there is no persistent object cache. With Redis holding transients, a duplicate write is one extra Redis command. Without it, each transient is two rows in the options table, the value and the _transient_timeout_ companion, and each miss writes both. Nine duplicate rebuilds become dozens of INSERT and UPDATE statements on the table that every uncached request in WordPress already reads from at least twice.

Only some keys deserve this work, and the ones that do are countable

Herd protection adds code, a coordination primitive and a new failure mode to every key it touches. Apply it where a rebuild is expensive relative to the traffic the key receives. A miss is countable: wrap the miss path in a counter keyed by cache key, send it somewhere cheap, and look at rebuilds per minute next to requests per minute. A key with a handful of rebuilds an hour does not need a lock. A key with one rebuild every five minutes and a two-second rebuild is a candidate, and a key whose rebuild count spikes when its TTL elapses is stampeding today.

The other half of the decision is what a stale value costs. If it is a slightly old list of posts, stale-while-revalidate is the cheapest fix, because it removes the only number that matters, which is how long a visitor waits. If it is a wrong charge, the lock is the only fix and the wait is the price of correctness. Write that sentence down next to the key, because the next person to touch the code will otherwise make the decision again from scratch.

add_action( 'template_redirect', function () {
    if ( ! isset( $_GET['expensive'] ) ) {
        return;
    }

    $built = 0;
    $value = wp_cache_get( 'expensive_value', 'myplugin' );

    if ( false === $value ) {
        $built = 1;
        $value = expensive_build();
        wp_cache_set( 'expensive_value', $value, 'myplugin', 300 );
    }

    header( 'X-Built: ' . $built );
    echo $value;
    exit;
} );