Build a WordPress Benchmark You Can Trust Before You Change Anything

A ticket says the product page "feels slow since the update". You have three candidate causes, two hours, and no way to tell which change helped. So you install a caching plugin, reload the page a few times, and it does feel faster. It also felt faster before you installed anything, on a warm browser, over a fast connection, at eleven in the morning.

Nothing in that sequence is a measurement. It is a sequence of impressions, and impressions cannot be compared: not against last week, not against staging, not against the version you just rolled back.

A benchmark that survives a code change needs three things fixed before the first request goes out. The data, the cache state, and the request itself. Fix all three and the numbers get boring, which is the point. Boring numbers tell you when something moved.

A benchmark you cannot repeat is not evidence

Request time is a sum. One part is your server working: bootstrap, the options table, plugin files, queries, template rendering. The other part is everything around it: DNS, the TCP handshake, TLS, the network between you and the host, and the transfer of however many bytes the response happens to weigh.

Only the first part changes when you change code. Measure time_total from a laptop on a hotel connection and both parts arrive in the same number, with the second one usually dominating. You will then spend a week optimising a query that was never the problem.

So the useful metric is the one that isolates the server's share, and the useful report names the conditions it was taken under. Making that number stable enough to compare is mostly a matter of removing the variables you did not mean to change.

Warm cache and cold cache are two different benchmarks

WordPress without a persistent object cache keeps cached values for the life of one request. Nothing is shared between two page views. Put Redis or Memcached in front of it and the picture inverts: the first request after a flush builds every entry the page touches, and each request after that reads them.

Those two requests differ by more than any plugin change you are likely to make. If you flush, measure, and compare against a baseline you took an hour earlier on a warm cache, you have measured the flush. The regression is real in the arithmetic and meaningless in the engineering.

Name the state and hold it fixed. A cold-cache run answers what the first visitor after a deploy pays. A warm-cache run answers what a settled request costs. Both are worth having, and neither can be compared against the other.

Without command line access to flush a cache, you can still benchmark. Write down that the cache state was uncontrolled, and keep every sample inside one short window so the state is at least consistent from the first request to the last.

One request is never a sample

Send the same request fifty times and you will not get fifty identical numbers. Workers get recycled, cron fires, a backup starts on the host, and the machine you share with forty other accounts has a bad minute. The distribution has a long right tail, and the tail is what customers complain about.

The mean is the wrong summary for that shape. One catastrophic sample drags it somewhere no user ever experienced. Report the median for the typical request and p95 for the slowest one in twenty, then put the sample count next to both, because a p95 computed from eight requests is a guess wearing a decimal point.

Throw away the first few requests. With a cold opcache, PHP compiles the files it needs on the first hit, and that cost belongs to the restart rather than to the page. Five warm-up requests followed by fifty measured ones is a reasonable default for a small install.

On a WordPress 6.7 install running PHP 8.3 with Redis on a 5,000-post dataset, cold TTFB started near 380 ms and settled around 240 ms once the object cache was warm, with run-to-run medians inside 15 ms of each other after that. The first figure reflects a cold cache rather than a regression, and putting it next to the second one without a label is how a benchmark gets misread.

Fix the dataset before you change any code

A benchmark compares two runs of the same site, so the site has to stay the same site. Content volume drives the numbers: menu lookups, term counting, related-item queries and feed generation all scale with the row count. If the baseline ran against 400 posts and the after-run against 900, you compared datasets and called it a code change.

Generate the content once from a deterministic script and keep it. Fixed titles give fixed slugs, and fixed body length keeps the postmeta and full-text work steady, so the only thing that varies between runs is the code.

#!/usr/bin/env bash
# Seed a fixed dataset. Run once, then leave the content alone while measuring.
set -euo pipefail

for i in $(seq 1 2000); do
  wp post create \
    --post_type=post \
    --post_status=publish \
    --post_title="Bench post $i" \
    --post_content="$(printf 'filler %.0s' $(seq 1 400))" \
    --porcelain > /dev/null
done

wp term create category "Bench category" --porcelain > /dev/null

Record the dataset size as part of the result rather than as background. A TTFB figure without a row count cannot be compared with anyone else's, including your own from six months ago.

The method belongs in the record next to the numbers. Which URL, how many iterations, which cache state, which commands produced the file. A setup that lives only in your shell history cannot be reproduced three weeks later, by you or by whoever reviews the change.

The fields worth capturing are short. Anything you cannot fill in is a variable you are not controlling.

Measure the server's share, not the network's

curl will hand you a timing breakdown if you ask for it. Two fields carry most of the value. time_starttransfer is when the first byte of the response is ready to go out, which is the application's time to first byte. time_total is everything, including the body transfer.

The connection fields are cumulative, and that detail catches people out. time_connect already includes time_namelookup, and time_appconnect includes both. So the server's share is time_starttransfer minus time_appconnect on HTTPS, or minus time_connect on plain HTTP. What is left is the wait for the application to produce that first byte.

Here is the loop I use for a local build. It writes one CSV line per request, never varies the URL, and discards a warm-up phase that would otherwise pay for a cold opcache.

#!/usr/bin/env bash
# Fifty measured requests against one URL. WordPress 6.7, PHP 8.3, Redis warm.
set -euo pipefail

URL="http://127.0.0.1:8080/product/bench-product-42/"
RUNS=50
WARMUP=5
OUT="ttfb-$(date +%Y%m%d-%H%M%S).csv"

FMT='%{time_namelookup},%{time_connect},%{time_appconnect},'
FMT+='%{time_starttransfer},%{time_total},%{http_code},%{size_download}\n'

# Warm-up requests are thrown away. They pay for a cold opcache.
for _ in $(seq 1 "$WARMUP"); do
  curl -sS -o /dev/null "$URL" > /dev/null
done

for _ in $(seq 1 "$RUNS"); do
  curl -sS -o /dev/null -w "$FMT" "$URL" >> "$OUT"
done

echo "wrote $OUT"

Read the file back with something that sorts, because the shape of the distribution is the result. This is the whole analysis step for a first pass. It ignores non-200 responses instead of counting them as fast pages.

python3 - "$OUT" <<'PY'
import statistics
import sys

rows = []
with open(sys.argv[1], encoding="utf-8") as handle:
    for line in handle:
        parts = line.strip().split(",")
        if len(parts) != 7 or parts[5] != "200":
            continue
        rows.append([float(value) for value in parts])

# Server-side wait: the application, with connect and TLS removed.
server = sorted(row[3] - row[1] for row in rows)


def nearest_rank(values, percentile):
    index = int(round(percentile / 100 * len(values) + 0.5)) - 1
    return values[max(0, min(index, len(values) - 1))]


print(f"samples        {len(server)}")
print(f"median server  {statistics.median(server) * 1000:.1f} ms")
print(f"p95 server     {nearest_rank(server, 95) * 1000:.1f} ms")
print(f"max server     {max(server) * 1000:.1f} ms")
PY

Print the sample count for the same reason you record the dataset size. If fifty requests go out and forty-one come back as 200s, the run has a problem that matters more than the timing does.

Four things still bend the result if you let them.

A laptop benchmark describes the laptop

Everything above produces a number that is comparable to other numbers taken the same way on the same machine. That is the entire claim, and it is a smaller claim than people usually want. It is not a prediction of production time to first byte.

Your laptop has different storage, a different CPU, no other tenants, and possibly a PHP build with a differently tuned opcode cache. A query that takes 4 ms locally can take 40 ms against a network-attached database, and the ranking of your three candidate fixes can flip with it. There is also no concurrency. A sequential loop never reproduces the queueing that appears when forty PHP workers contend for the same database.

Use the method to decide what to change, then run the same script against production hardware at the same dataset size before you claim a win. If production is off limits, keep the claim small: the median moved by this much, under these conditions, on this machine.

What to do on Monday

1. Write one line into the ticket: dataset size, WordPress version, PHP version, object cache, cache state.
2. Generate the fixed dataset and stop editing content while you measure. A draft post published mid-run changes the data under the test.
3. Run the loop with a warm-up phase, keep the CSV, and keep the baseline CSV too. A baseline that only exists as a screenshot gets lost.
4. Report median and p95 for server-side time, plus the status mix and the response size. Skip the mean entirely.
5. Change one thing, then re-run both cache states and compare like with like.
6. Re-run on production hardware before you tell anyone the site is faster.