Idempotency Keys for a POST Endpoint That Must Survive Retries

A client posts an order and waits. Ten seconds later the connection drops with no response, no status code and no explanation. The client now has two options and both look bad: abandon a request that may have created an order, or send it again and risk creating a second one.

The retry is the correct choice. A timeout means the client stopped waiting, not that the server stopped working. The write may have committed with the response lost on the way back, or the request may never have reached PHP at all. Those two cases are indistinguishable from where the client sits, and only one of them recovers by retrying. A client that refuses to retry turns a two-second network blip into a missing order.

So the retry happens, and the server has to absorb it. That is what an idempotency key buys you. It does not stop the duplicate request from arriving. It makes the duplicate return the first request's answer without running the work again. On a checkout that talks to a payment provider, that is one authorisation instead of two.

A timed-out client cannot know whether the write committed

HTTP gives a client exactly one signal about the outcome: the response. When the socket closes before that response arrives, the outcome becomes a three-way unknown. The request may have been lost before it reached the server. It may have arrived, failed, and returned a 500 that the client never saw. Or it may have succeeded completely, with the response lost in the last few milliseconds.

Idempotency is the property you add to close that gap. An operation is idempotent when running it twice leaves the system in the same state as running it once. A PUT that replaces a resource behaves that way, and so does a DELETE. A POST that inserts a row does not: every call adds another row.

The client generates the key, and the server enforces it

Only the client knows that two requests are the same intent, so the key has to come from the client. Deriving it on the server from the payload works until two customers order the same product inside the same minute, at which point the second order gets discarded as a duplicate of the first.

The convention is a request header, usually Idempotency-Key, carrying a value the client creates once per logical attempt. A UUID is fine. What matters is that the value exists before the first attempt and is reused on every retry of that attempt.

A new key on every HTTP request protects nothing, and it is the most common implementation mistake I run into. It normally comes from generating the key inside a retry helper, which runs once per attempt instead of once per operation.

key=$(uuidgen | tr 'A-Z' 'a-z')

# Reuse $key for every retry of this order, and for nothing else.
curl -sS -X POST https://example.com/wp-json/wps/v1/orders \
  -H "Idempotency-Key: $key" \
  -H 'Content-Type: application/json' \
  --data-binary @order.json

Validate the value before you use it. Accept a known character set and a bounded length, and reject anything else with a 400 before the raw header can reach a query.

Storing the response is what turns a duplicate into a replay

A table that stores only the key stops the second write and hands the client a 409 with nothing else. That is worse than the duplicate it prevented, because the client cannot tell "already done" apart from "someone else is doing this right now". Store the outcome with the key: the status code and the serialised body, returned verbatim the next time that key arrives.

The table needs one thing the application layer cannot supply, which is a unique index on the key. Everything else follows from that index.

CREATE TABLE wp_idempotency_keys (
  id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  idem_key VARCHAR(64) NOT NULL,
  route VARCHAR(191) NOT NULL,
  request_hash CHAR(64) NOT NULL,
  response_code SMALLINT UNSIGNED NOT NULL DEFAULT 0,
  response_body LONGTEXT NULL,
  created_at DATETIME NOT NULL,
  PRIMARY KEY  (id),
  UNIQUE KEY idem_key (idem_key),
  KEY created_at (created_at)
);

The unique index is the load-bearing part of that schema, because it makes the database the arbiter of who owns a key. The next section depends on it. response_code defaults to zero so a claimed key with no stored response is distinguishable from a finished one.

If you create the table with dbDelta(), keep the formatting it expects: two spaces after PRIMARY KEY, one column per line, and KEY rather than INDEX. dbDelta parses the statement as text and quietly skips definitions it does not recognise, which is how people end up with a table that has no unique index and duplicates that only appear under load.

The handler below claims the key before doing any work, then records the response. It is a plain register_rest_route() callback with a real capability check, because an idempotency key is not authentication.

add_action( 'rest_api_init', function () {
    register_rest_route(
        'wps/v1',
        '/orders',
        array(
            'methods'             => 'POST',
            'callback'            => 'wps_create_order',
            'permission_callback' => function ( WP_REST_Request $request ) {
                return current_user_can( 'edit_shop_orders' );
            },
        )
    );
} );

function wps_create_order( WP_REST_Request $request ) {
    global $wpdb;

    $key   = (string) $request->get_header( 'idempotency_key' );
    $table = $wpdb->prefix . 'idempotency_keys';

    if ( ! preg_match( '/^[A-Za-z0-9_-]{16,64}$/', $key ) ) {
        return new WP_Error( 'wps_bad_key', 'Send an Idempotency-Key header.', array( 'status' => 400 ) );
    }

    // Fingerprint the method, the route and the exact bytes received.
    $hash = hash( 'sha256', 'POST /wps/v1/orders ' . $request->get_body() );

    // Claim first. The unique index decides who owns the key.
    $wpdb->suppress_errors( true );
    $claimed = $wpdb->insert(
        $table,
        array(
            'idem_key'      => $key,
            'route'         => '/wps/v1/orders',
            'request_hash'  => $hash,
            'response_code' => 0,
            'created_at'    => current_time( 'mysql', true ),
        ),
        array( '%s', '%s', '%s', '%d', '%s' )
    );
    $wpdb->suppress_errors( false );

    if ( ! $claimed ) {
        $row = $wpdb->get_row(
            $wpdb->prepare( "SELECT request_hash, response_code, response_body FROM {$table} WHERE idem_key = %s", $key )
        );

        if ( ! $row ) {
            return new WP_Error( 'wps_in_progress', 'Retry shortly.', array( 'status' => 409 ) );
        }

        if ( $row->request_hash !== $hash ) {
            return new WP_Error( 'wps_key_reused', 'This key was used for a different request.', array( 'status' => 422 ) );
        }

        if ( 0 === (int) $row->response_code ) {
            return new WP_Error( 'wps_in_progress', 'Retry shortly.', array( 'status' => 409 ) );
        }

        return new WP_REST_Response( json_decode( $row->response_body, true ), (int) $row->response_code );
    }

    $response = rest_ensure_response( wps_do_the_work( $request ) );

    $wpdb->update(
        $table,
        array(
            'response_code' => $response->get_status(),
            'response_body' => wp_json_encode( $response->get_data() ),
        ),
        array( 'idem_key' => $key ),
        array( '%d', '%s' ),
        array( '%s' )
    );

    return $response;
}

The same key and the same body return the same order. The second call does no work.

POST /wp-json/wps/v1/orders HTTP/1.1
Idempotency-Key: 7f3a1c9e-2b44-4d8a-9f10-5c6e8b2d4a71

HTTP/1.1 201 Created
{"id":8123,"status":"pending"}

# the client times out, keeps the key, and sends the same bytes again

POST /wp-json/wps/v1/orders HTTP/1.1
Idempotency-Key: 7f3a1c9e-2b44-4d8a-9f10-5c6e8b2d4a71

HTTP/1.1 201 Created
{"id":8123,"status":"pending"}

How long the row has to stay

Retention is bounded below by the client's own retry policy. A sender that retries for twenty-four hours needs a window longer than twenty-four hours, because a late retry that finds no key writes the duplicate you were preventing. Forty-eight hours survives a weekend outage.

The storage bill is real. On a store creating 50,000 orders a day and storing 2 KB of response per row, that is roughly 100 MB a day written into the table. At 48 hours retention the steady state is about 200 MB before index overhead, so the window is a disk budget with a purge job attached.

Purge completed rows older than the window on a schedule. Leave rows with response_code at zero alone until you have decided what an abandoned claim means, because deleting one turns a stuck request into a fresh duplicate.

The same key with a different body is a client bug

Replay on the key alone and a client that reuses a key for a different payload receives the old response and believes its new request succeeded. That failure is quieter than a duplicate: the client logs a success, the server logs nothing unusual, and the second piece of work never exists.

The fix is the fingerprint in the code above. Hash the method, the route and the raw body together, store the hash with the key, and compare it on every duplicate. Putting the route inside the hash means a key reused across two endpoints is caught by the same check.

When the fingerprints differ, do not overwrite the stored response and do not process the request. Return 409 or 422 with a message that names the header, and never accept the reuse silently. The client's mistake is cheap to fix. Silent state corruption is not.

Failure mode	What a key-only table does	Correct handling
Client retries after a timeout, and the first request committed	Blocks the second write and returns 409	Replay the stored status and body for that key
Same key, different body	Replays the old response and skips the new work	Compare fingerprints, return 422, leave the stored response untouched
Two requests with one key arrive together	Both read an empty table and both write	Let the unique index pick a winner; the loser reads or waits
The process dies after the work and before the response is stored	The key exists with no response, so every retry gets 409	Claim before the work, mark complete after, alert on claims older than a few minutes
The row is purged while the client is still retrying	The key is gone, so the retry writes a second order	Keep retention longer than the client's longest retry window
The client mints a new key per attempt	No duplicate is ever detected	Generate the key once per logical operation and reuse it

Concurrency is the unique index's job, not yours

Retrying is not the only way one key arrives twice. A client with a three-second timeout and a slow first response can fire the second request while the first is still running. A load balancer retrying an upstream connection does the same thing. Neither request finds a stored response.

The naive implementation reads first: select the row, and if it is missing, insert it. There is a gap between the select and the insert, and the gap is exactly wide enough for the other request. Under normal traffic you never see it. Under a spike you see it routinely, which is why this bug turns up on the worst possible day.

Let the database decide instead. Insert the claim row unconditionally and read the result. The unique index lets one insert through and rejects the other with a duplicate-key error, so there is no gap to lose.

In the handler above, $wpdb->insert() returns false for the loser, and that false is read as "somebody else owns this key", not as a write failure. The loser loads the winner's row. If the winner has finished, the loser returns the stored response. If the winner is still working, response_code is still zero and the loser gets a 409 with a retry hint.

Suppress the database error around the insert unless you want a duplicate-key notice printed on every retry. Make the follow-up read go to the primary as well. On a site with a read replica, loading the winner's row from a replica that has not caught up reintroduces the gap you just closed.

You could hold the loser inside SELECT ... FOR UPDATE instead, but that keeps a transaction open for the length of the order creation and adds a lock timeout to tune. Returning 409 with a retry hint fails visibly instead of parking a PHP worker.

Where idempotency keys stop working

The key makes the boundary of one HTTP request safe to cross twice. It does not make the inside of that request atomic. A handler that charges a card, writes an order row and schedules a shipment can still die between the second step and the third. When that happens the key row is claimed, the client gets a 409 on every retry, and the order sits half-built. No amount of key handling fixes it. You need a state machine that records which step it reached, with each step safe to repeat on its own, or a database transaction covering the whole thing, which is rarely available when one step is a remote call.

The key table is also a shared dependency with a failure mode of its own. If it is unreachable you have two bad options: process without a claim and accept duplicates, or refuse and accept an outage. Refuse with a 503 for anything that moves money or ships goods.

Two smaller limits belong in the same list. Keys do not help when the client mints a new one per attempt, or when it stores the key only in memory and loses it on a crash. And if the stored response contains a short-lived token or a signed URL, keeping it for forty-eight hours means keeping something sensitive in a table that lands in your backups.

What to do on Monday

1. Add the key requirement to one endpoint that creates something. Not all of them, and not the read paths.
2. Create the table with the unique index, and verify the index exists with SHOW INDEX FROM wp_idempotency_keys rather than trusting the activation code.
3. Move the key generation to the caller, once per operation, and delete every retry helper that generates its own.
4. Force a timeout in staging and send the same key twice. Confirm the second response carries the first response's ID.
5. Send the same key with a changed body and confirm you get a 422 rather than a replay.
6. Add the purge job and a monitor for claimed rows older than ten minutes. Those are the requests that died mid-flight.