The builder never grades its own homework

Checked 22 Sep 2026 · By Luke Czak

ArticleOpinionFree to read

Every AI-written change in my estate gets a fresh reviewer session with no memory of building it; using the same engine does not mean using the same session. Self-certification is the failure mode agentic development keeps rediscovering the hard way.

I used to let the same session that wrote a change also decide whether the change was good. It felt efficient. The model had full context, it had just read every file it touched, and asking it to review its own diff seemed to avoid a second round of reading. It also produced reviews that were almost worthless, because a model reviewing its own output is grading its own homework, and it grades generously in exactly the places it was wrong.

The failure is not that the model lies. It is that a review is only as independent as the assumptions it is willing to question, and a session carries forward every assumption it made while writing the code. If it decided a null case could not happen, that belief does not get re-examined five minutes later by the same context window. It gets carried into the review as settled fact, because from the inside it still looks settled. The bug and the blind spot that let it through share an author.

So the rule in my pipeline now is plain: the session that builds a change never gates it. A build runs on one CLI, in its own worktree, with its own session. The gate is a fresh session, which may use the same engine, given the candidate and acceptance criteria without the conversation that produced the change. It has to work out what the code does from what is actually on disk, which is exactly the position a real reviewer is in and exactly the position the builder cannot occupy.

This is more expensive than self-review. The gate has to re-read the surrounding code cold rather than lean on context the builder already paid for, so a good deal of the reading gets done twice. I still keep it, because the alternative is a rubber stamp that happens to be well-written. I have watched a self-reviewing session mark its own race condition as fine, because the explanation it gave for why the race was safe was the same flawed reasoning that let it write the race in the first place.

What surprised me is how often the independent gate disagrees with a fix that looked obviously correct on the page. Not because the code is wrong in some subtle algorithmic sense, but because the reviewer asks a question the builder never had reason to ask: what happens when this runs against the acceptance criteria the builder was actually given, not the ones it remembers giving itself. Those two lists drift apart more than I expected, and only a session with no stake in the first draft catches the drift.

The pattern generalises past code. Anywhere a system produces an artefact and is then asked whether the artefact is acceptable, the answer is compromised by the fact that the producer wants to have succeeded. Humans have peer review, second opinions, and separate sign-off chains for exactly this reason, and none of that is new. What agentic development adds is that the temptation to skip it is much stronger, because the same session doing both jobs is right there, already primed, and asking it to mark its own work feels like it should be fine. The temptation is strongest exactly where the stakes are highest, which is the worst possible correlation.

It is not fine, and the cost of proving that to myself was a handful of changes that shipped with defects a separate reviewer could have challenged before release. I would rather pay the token bill for an independent pass every time than find out again, in production, exactly which assumption the builder never thought to check. The bill is predictable. The alternative is not.

Comments (0)

Sign in to comment.