Write down what “done” means before an agent touches the code, then grade the running result against that list, not the diff.
I stopped letting an agent start writing code without first writing down what “done” means. Not because I doubted its intent, but because an agent asked whether it finished a task will almost always say yes, and the code it produced will often support that answer even when the answer is wrong. The check has to live somewhere outside the code, or it isn’t really a check, it’s a self-report.
The habit is mechanical and I apply it before a single line changes. I write “this task is done when” and list every criterion as a plain sentence, numbered, specific enough that a stranger could grade it without reading the diff. Not “the login works” but “a user with valid credentials reaches the dashboard and sees their own name in the header, on the first attempt, within a couple of seconds.” The precision is the point — a vague criterion can be satisfied by almost anything, which means it isn’t a criterion at all, it’s a mood.
Writing the list first does two things. It forces me to decide what I actually want before I’ve seen any code, which is the only point in the process where my judgement isn’t already contaminated by what got built. And it gives the agent a fixed target. I’ve watched agents quietly redefine “done” mid-task to match whatever they happened to produce — not out of dishonesty, just drift, the same way a person’s memory of a plan bends towards what they actually did. A written criterion makes that drift visible instead of invisible, because there’s a fixed sentence to check the outcome against, dated before the work started.
The list works the other way too, on me, and this is the part I underestimated when I started doing it. Without a criterion written in advance, the temptation after seeing a plausible result is to grade the outcome rather than the target — to quietly decide the result is good enough, which is a milder version of the same drift I’m trying to catch in the agent. A dated criterion removes that temptation, because moving the goalposts after the fact is visible as exactly what it is: rewriting the test to match the answer, rather than checking the answer against the test.
The part most people skip is verification, and it’s the part that actually matters. Reading the code and deciding it seems sound is not verification — it’s a guess dressed up as diligence, and it fails silently in exactly the cases where it matters most. Verification means running the thing against each criterion, one at a time, and writing down what actually happened: not “the login flow works” but the real screen state, the real timing, the real text in the header. If I can’t verify a criterion — no environment to run it in, no way to observe the state — I say that plainly, rather than letting silence stand in for a pass.
I’ve had criteria that read as trivially satisfied from the diff and turned out false the moment I actually ran the thing. A redirect that fired before the database write had committed, so the next page loaded stale data. A form that validated cleanly in the browser but accepted anything at all once it reached the server. Neither would have shown up from a careful read of the code — both showed up in under a minute against a checklist that existed before the code did, because the checklist asks about behaviour, and the code only tells you about intent.
The list changes the shape of disagreement too. Instead of an argument about whether a task is finished — which tends to become an argument about whose reading of the code is more careful — there’s a criterion with a status next to it, and the status has evidence attached. Nobody has to trust my interpretation of the diff, because the interpretation isn’t what’s being trusted. The running system is. That’s a small shift in where the authority sits, and it’s the one that actually holds up under pressure to ship.