Testing Generated Code, and Knowing When to Throw It Away

The code arrives, it activates, the block renders, and the agent's summary says it tested everything. At this point the temptation is to keep reading the summary, and the summary is the least reliable artefact in the exchange. The install is the reliable one, and the question this lesson answers is what to do with it: how to run the code in a way that can fail, and how to decide whether the result is a scaffold you keep or a draft you delete.

There is also a decision at the end of this lesson that has nothing to do with tools. The recorded run behind this course produced a fourteen-file plugin that worked, and the honest conclusion from it is narrower than the headline. One run, one model, one task type, and a task sitting on top of the most heavily represented WordPress pattern in existence. That is worth something, and knowing what it is worth is the skill this module has been building.

The first thing to do with generated code is not to rebuild it

Both of the toolchain's green lights are compatible with broken output, and the two failures have different consequences. A successful wp-scripts build means webpack resolved your imports and emitted files; as Module 6 lesson 1 showed, it externalises every @wordpress/* request to a wp.* global and never checks that the export exists, so a build can succeed for an import that throws at render. A clean php -l means the file parses, which is a statement about syntax and not about whether any function it calls exists.

So run the code. Activation, a command, and a fetch over HTTP, in that order, because each one catches a different layer and each one is cheap.

wp plugin activate acme-notice
wp acme-notice list
curl -s https://acme.test/?p=4 | grep -c "wp-block-acme-notice-notice"
tail -5 wp-content/debug.log

$ wp theme list --path=/srv/http/acme.test
name	status	update	version
acme-studio	active	none	1.0.0
twentytwentyfive	parent	none	1.5

$ curl -s -o page.html -w "HTTP %{http_code}\n" https://acme.test/
HTTP 200

$ tail -c 120 page.html
<a href="https://wordpress.org" rel="nofollow">WordPress</a>		</p>
			</div>
		</div>
</footer>
</div>

The first three checks have expected shapes you can recognise. Activation prints a success line or an error naming the header that is wrong. A plugin's own WP-CLI command prints its rows, or returns a non-zero exit code with the reason, or fails with "is not a registered wp command", which means the command class is not being loaded. The fetch either contains the block's class name or it does not, and a 200 with a truncated body means the fatal happened after the response started, which is the shape from Module 5 applied to new code.

Four checks, and one of them is not a pass or fail. The activation proves the plugin's headers and bootstrap are valid. The command proves the plugin's own surface works and that its data layer returns something. The fetch proves the front end renders and the assets enqueued. The debug log is where the failures that did not stop the response appear, and it is the check most often skipped, because a site with deprecation notices all over it looks identical to a clean one from the outside.

The outputs below are from the install this lesson was written on, which is the child theme from Module 5 rather than the plugin being described, so read them as the shapes to expect instead of a transcript of the acme-notice run.

Mutation is how you find out whether a test can fail

Passing tests are only evidence if they can fail. The quick test of a test is to break the thing it covers on purpose, in one place, and watch what happens. An agent's own verification almost never survives this, because the verification was written alongside the code and encodes the same assumptions.

Take the escaping claim. The agent reported that a hostile colour value falls back to a default. The mutation is to remove the validation and leave the escaping, which is exactly the mistake it might have made in the first place.

<?php
// Before: validated, so a hostile value cannot reach the attribute.
$colour = sanitize_hex_color( $attributes['backgroundColor'] );
$colour = $colour ? $colour : '#fff8e5';

// Mutation: keep the escaping, drop the validation.
$colour = esc_attr( $attributes['backgroundColor'] );

$ wp eval-file render-with-mutation.php
<div style="background-color:red;background:url(//evil.example/pixel.png)" class="acme-notice">

The mutated code renders the injection, which means the test that passed before and fails now was a real test. If the mutated version had still produced the safe output, the test would have been measuring the wrong thing, and the green result from the first run would have been decoration. This is a two-minute exercise and it is the difference between a verification claim and a verification.

Two mutations are worth running on almost any generated plugin. Remove the capability check from an admin action and confirm something fails, then restore it. Change one branch condition in the render callback and confirm the front end changes. Anything that stays green under both is not covered by a test, whatever the summary said.

Keep the scaffold, throw away the invention

Generated output is usually a mix, and the useful decision is made per part rather than per file. There are three kinds of thing in a typical agent delivery, and they deserve different treatment.

Kind	What it is	What to do
Scaffold	Directory structure, block.json, the readme skeleton, the build config, the licence header	Keep, then read once. It saves time and it is the part with the most prior art.
Logic	The render callback, the query, the capability check, the escaping	Keep only what you have run and mutated. Rewrite anything you cannot explain line by line.
Invention	Helper functions you did not ask for, an abstraction layer, a settings page nobody requested, caching that was not in the brief	Delete first, reinstate only against a requirement. This is the over-architected code the guidelines name.

The third row is where the hours go, and it is the one thing the tooling will never flag. Over- architected code passes every check, because it works. It is also the part an agent adds most freely, and it is the part that makes the next change harder for the next person, which is now you. Delete it while the diff is small and the reason is still in your head.

Say what the change is not

One instruction prevents more rework than any other, and it is not about verification. It is about scope. An agent asked to change one line in a file will sometimes restructure the file, because restructuring is a plausible reading of "make the footer say the slogan". The instruction that stops it is to name the files it may touch and the behaviour that must not change.

Task: make the footer render the client slogan.

Allowed to change:
- wp-content/themes/acme-studio/inc/footer.php

Must not change:
- any file under wp-content/themes/twentytwentyfive/
- the render order of the existing footer sections
- the escaping functions already applied to each value

Report: the diff, the command you ran, and its output.

Three lines of scope and one line of evidence. The scope line does the work that review cannot, because review happens after the rework has been done. The evidence line is what makes the report checkable, and a report without it is a claim rather than a result.

What six modules add up to

The tools in this course will be replaced, and several of the pricing pages in the brief are already out of date. What has not changed across any of the modules is where the truth lives. Every real check in this course was a command run against a WordPress installation: the symbol probes, the fatal reproduction, the render over HTTP, the hostile input fed through real kses and real sanitising functions, the bundle inspected after a green build. Everything else was reading, which is useful for forming a hypothesis and useless as evidence.

That gives one working rule, and it is the rule to keep when the models change. Treat agent output as a pull request from a contributor who has never used WordPress. Believe the API only after a probe has confirmed it, and believe a test only after you have seen it fail. Disclose the assistance, keep the scaffold, delete the invention, and say what you did not test. Nothing in this course worked because the model was good. It worked because the install could say no.

Show how to grade the answers, and what the sentence should contain

The mutation in step 2 is the part people get wrong, usually by mutating something the tests do not cover and concluding the tests are worthless. Mutate the claim, not a random line. If the agent claimed the capability check blocks non-editors, remove the check and make an authenticated low-privilege request; a fetch as an anonymous visitor proves nothing, because anonymous visitors fail the check for reasons unrelated to your code.

The scaffold, logic and invention split usually surprises people in the same direction. The scaffold is better than expected, because structure is the part with the most public examples and the least judgement in it. The invention is larger than expected, because adding a settings page, a cache and an abstraction over WP_Query is what a plausible answer looks like when nobody has told the agent what the requirement is. Logic sits in the middle and is the only part that needs a run to evaluate.

The final sentence should name a specific thing in each half. Something adjacent to: the agent was right about the plugin structure, the block.json metadata and the build config, and the settings page, the transient cache and the helper class that wrapped two function calls were deleted because nothing asked for them. A sentence that says the agent was helpful on balance has graded nothing.

The reason to write the sentence at all is that it is the transferable part. The structure is what you will accept next time without reading it twice, and the invention is what you will ask the agent not to produce next time, in the scope line, before it starts.