Transients, Cache Keys and Invalidation

Every site has one. A plugin caches a category listing with set_transient(), the listing is wrong after an editor moves a product, and the developer adds a delete_transient() call to the save hook. It works, until the day somebody imports products in bulk, and the listing is wrong again for a reason nobody can reproduce.

The reason is that set_transient() is being used as if it were a cache with an invalidation plan, and a transient is a value with a timer. A timer answers "how long may this be wrong", which is a real question and not the one the plugin needs answered. The other question is "when the data changes, how does the cached value find out", and that is a key design decision rather than a function call.

Four decisions follow: where the value is stored, which mechanism is responsible for correctness, how the key is built, and what happens to a popular entry when it is rebuilt. The measurements come from the lab install in the previous two lessons.

An expiring transient is two option rows, not one cached value

Start with what the API actually does, because most of the surprises live there.

set_transient( 'm3_live', array( 'payload' => 'value' ), 900 );
set_transient( 'm3_forever', array( 'payload' => 'value' ) );

Without a persistent object cache, the first of those writes two rows into wp_options, and the second writes one:

_transient_m3_live              autoload=off   18 bytes
_transient_timeout_m3_live      autoload=off   10 bytes   (a unix timestamp)
_transient_m3_forever           autoload=on    18 bytes

The timeout row is a separate option holding time() + $expiration, which is how get_transient() can answer "is this still valid" without touching the object cache. Both rows are non-autoloaded when there is an expiry, so neither joins the options blob, and the read path pays a query. On the lab install a live read from a fresh process cost one query on the default backend.

Attach a persistent object cache and the same call takes a different path entirely. Core checks wp_using_ext_object_cache() in get_transient(), set_transient() and delete_transient(), and when it is true the value is stored in the transient cache group with the expiration passed to wp_cache_set(). No option rows are written, the timeout row never exists, and the read costs zero queries. On the lab install, after attaching a persistent drop-in, the same live transient read issued no queries at all and left no rows in wp_options.

Both paths are correct, and they differ in ways that matter later:

That third row surprises people during an incident. Flushing the object cache to clear a bad transient works on one install and does nothing on another, and the difference is this branch rather than anything in the plugin.

A TTL bounds how long a value can be wrong; invalidation is what makes it right

These are two mechanisms with opposite failure shapes, and it is worth being able to say which one you are relying on in any given line of code.

A TTL is a promise about age. A value with a 900-second expiration is never permanently wrong and is always wrong for up to 900 seconds after the data changes. Its failure is bounded and knowable: you chose the number, so you chose the window during which a customer may see a stale price or a count that includes a sold-out item.

An invalidation event is a promise about the source. If every write path that changes the underlying rows deletes or versions the cached value, the value is exactly correct until the one write path you forgot, and then it is wrong with no upper bound at all. That is the shape that produces a bug report three weeks later with no reproduction steps.

So the useful question is not "TTL or invalidation" but "what is my ceiling if the invalidation is incomplete", and the answer should be a number you are willing to defend. A category product count can be wrong for fifteen minutes. A stock count on a flash-sale item can be wrong for as long as it takes to issue a refund.

The key is the invalidation plan, so design it first

Everything below follows from one habit: build the key from a seed that contains every input the value depends on, and build it in one function that the read path and anything that deletes it both call. A key assembled at the delete site differs by one character from the key the read wrote, and a delete that targets the wrong key reports success.

Three of the possible seeds are worth naming because they are easy to forget:

$seed = array(
    'ids'      => array_map( 'intval', $product_ids ),
    'currency' => $currency,
    // The marker for the group that holds the rows this value is derived from.
    'posts'    => wp_cache_get_last_changed( 'posts' ),
    'meta'     => wp_cache_get_last_changed( 'post_meta' ),
);

$key = 'price-table:' . md5( wp_json_encode( $seed ) );

wp_cache_get_last_changed( $group ) reads the group's marker entry and creates it if it is absent, and core moves that marker from the write path: wp_cache_set_last_changed( 'posts' ) is what a post or meta write calls to say "keys built inside this group are now stale". Putting the marker in the seed means the key changes when the data does, without anything having to enumerate keys or remember to delete them.

The failure mode is a seed that is incomplete, and it is a worse bug than a stale entry, because an incomplete seed can serve one context's value to another. The day somebody adds a dependency on the customer's tax location or on a coupon's state, that input has to enter the seed, or the cache will confidently return the wrong price. Keeping the seed next to the query it wraps makes that visible in review; keeping it in a config array three files away makes it invisible until it is a support ticket.

Bump a marker, delete keys, or version the key

Almost every invalidation you will write is one of three shapes, and each one has a characteristic bug. The invalidation article goes through the reasoning; what follows is the practical version, with the failure you will actually see in monitoring.

Bump a shared marker. One write to last_changed and every key derived from it stops matching. This is what core does, and it is the right default when the group is small and writes are rare. Its bug is not staleness, it is a cache that stops working: register the bump on an action that fires on every page view, or on a meta key that a background job writes every minute, and the marker moves so often that the cache never hits. That shows up as a hit rate near zero on a group that used to be warm, and the fix is at the hook rather than in the cache.

Delete specific keys. Precise, and it is the strategy that fails silently. One product save can touch a product key, three category keys, a price-range facet and a count. Every key you forgot stays wrong with nothing watching it, and the delete you did write may not have done anything. Deletion reports itself through a return value that means several things at once: true when a key existed and was removed, false both for a key that was already gone and for a key name that was never used. Changing product:42 to product_42 produces no error, no warning and no effect, which is why the key string belongs in one function that the read and the delete both call.

Version the key by its inputs. Nothing is deleted; the key changes when an input changes and the old value expires on its own. It has no hook to register and no key to forget. Its cost is the seed contract above, and its advantage is that a missing input is a review comment on one function rather than a search through every write path in the plugin.

The practical rule: use the marker where the group is small and writes are rare, use a versioned key where the group is large or written constantly, and treat delete-a-key as a deliberate choice for the handful of entries where the other two do not fit.

Audit for rows nothing will ever clean up

A transient with an expiry is removed when it is next read after the timeout, or by a cleanup job. A transient with no expiry is an ordinary option that lives until something deletes it, and on a site where plugins create those and disappear, the rows accumulate. This query finds value rows whose timeout row is missing, which is the shape both of a non-expiring transient and of a leaked one:

SELECT o.option_name,
       o.autoload,
       LENGTH(o.option_value) AS bytes
FROM wp_options o
WHERE o.option_name LIKE '\_transient\_%'
  AND o.option_name NOT LIKE '\_transient\_timeout\_%'
  AND NOT EXISTS (
      SELECT 1 FROM wp_options t
      WHERE t.option_name = CONCAT( '_transient_timeout_', SUBSTRING( o.option_name, 12 ) )
  )
ORDER BY bytes DESC
LIMIT 20;

The result needs interpretation rather than a script. On a freshly installed WordPress 7.0.7 the query returns two rows, both written by core itself: _transient_wp_core_block_css_files at 22,750 bytes and _transient_wp_styles_for_blocks at 11,356 bytes. Both are non-expiring by design and both are autoloaded, because core wants them loaded with the options blob rather than fetched again. Those are not garbage and deleting them costs a rebuild.

The rows worth acting on are the ones that are large, that nothing reads, and that belong to a plugin you can identify. Grep the plugin directory and your own code for the transient name without the _transient_ prefix before you delete anything, because the name in the options table is not the name in the code. A row that nothing reads gets deleted; a row that something still reads gets a fix at the source, and if it is storing a table in one option value, that fix is the custom-table conversation from module 2.

Check the other direction too, for timeout rows whose value row is gone. Both are cheap and safe to clean up, which makes this a useful job to run from system cron weekly and log the total bytes.

The popular key fails when it expires, not when it is wrong

Invalidation and expiry share one more property: at the moment a key stops being valid, the next requests all find nothing and all rebuild. That is the thundering herd, and it is the reason the entry that matters most is the one that gets rebuilt most often.

The mechanism is easy to demonstrate even though real concurrency is not. Start eight PHP processes that each wait for a file and then read the same missing key. With no coordination, the lab run produces eight rebuilds. With wp_cache_add() as a lock, one rebuild and seven readers that find the lock held:

// wp_cache_add() returns false when the key already exists, which is what a lock needs.
if ( ! wp_cache_add( $key . ':lock', 1, 'myprefix', 5 ) ) {
    // Somebody else is rebuilding. Serve the stale copy if there is one.
    return $stale;
}

try {
    $value = myprefix_rebuild( $key );
    wp_cache_set( $key, $value, 'myprefix', 900 );
    return $value;
} finally {
    wp_cache_delete( $key . ':lock', 'myprefix' );
}

Three details decide whether that snippet helps or hurts. The lock key has to be built from the same function that builds the cache key, or it locks something nobody else is waiting on. The lock needs a TTL shorter than the worst-case rebuild, so a worker that dies mid-rebuild cannot wedge every reader behind a lock that is never released; with the lock's TTL shorter than the rebuild, two workers may rebuild in parallel and the read path still cannot hang. And the value needs a hard TTL longer than its freshness window, so there is a stale copy to serve while one worker refreshes.

A persistent cache changes the size of that event rather than its shape. On the default backend every request is a cold start, so the herd is constant and cheap: the rebuild work happens anyway. Attach a persistent cache and the herd gathers at rare moments instead, which makes it less frequent and more dangerous, because the rebuild now happens at the moment a key that thousands of requests per minute depend on disappears.

function myprefix_listing_key( array $term_ids, string $orderby ): string {
    $seed = array(
        'term_ids' => array_map( 'intval', $term_ids ),
        'orderby'  => $orderby,
        'posts'    => wp_cache_get_last_changed( 'posts' ),
        'terms'    => wp_cache_get_last_changed( 'terms' ),
    );

    return 'listing:' . md5( wp_json_encode( $seed ) );
}

function myprefix_listing( array $term_ids, string $orderby ): array {
    $key   = myprefix_listing_key( $term_ids, $orderby );
    $found = null;
    $value = wp_cache_get( $key, 'myprefix', false, $found );

    if ( $found ) {
        return $value;
    }

    $value = myprefix_query_listing( $term_ids, $orderby );
    wp_cache_set( $key, $value, 'myprefix', 15 * MINUTE_IN_SECONDS );

    return $value;
}