The First Draft Is Supposed to Be Wrong
4 min read
Not long ago, one of my models wrote a batch of new code with three real defects in it: a server-side request forgery hole in a file-sourced allowlist, a build config that was inverted so it would have shipped the wrong artifact, and a set of OAuth defaults that were quietly wrong. None of it reached production. It shipped clean — because a different model, whose only job was to try to break the first one's work, caught all three before I ever looked.
That's not a failure story. That's the system working exactly as designed.
The author is not the shipper
I run my development work as a two-role pipeline, and the roles are deliberately different models with different jobs.
Fable 5 is the author. It writes the hard first draft — a new worker, a whole skill library, a migration. It's fast and it's strong, and its first pass is often wrong in the specific, real ways above. That's fine. That's expected. I'm not paying it to be clean; I'm paying it to get a complete, serious draft onto the page faster than I could.
Opus 4.8 is the editor and the shipper. It reviews the draft, runs an adversarial pass explicitly trying to refute or break it, fixes what it finds, and only then merges and deploys. Generate → adversarially verify → ship. The system works not because the generator is clean, but because nothing gets to production without surviving a model that was told to distrust it.
Generation got cheap. Trust didn't.
Here's the shift almost everyone gets backwards. The instinct is to optimize the generator — pick the model that writes the fewest bugs, prompt it harder, chase a cleaner first draft. But generation is now the cheap part. I can produce far more code than I can responsibly read. The scarce resource isn't good first drafts. It's trust — knowing the thing is actually safe to ship.
So I stopped optimizing the draft and started optimizing the verifier. A generator that's wrong 30% of the time, behind a verifier that catches 99% of what's wrong, beats a generator that's wrong 5% of the time that you trust blindly. The second one ships the 5%. The first one ships nothing bad, ever, as long as the verifier holds.
Verification is adversarial or it's theater
The word doing the work up there is adversarial. A verifier that reads code and says "looks good" is worthless — it pattern-matches to plausible and moves on. A verifier earns its keep only when its job is to refute: assume the draft is broken, and go prove it.
That principle shows up in three places in my stack, and it's the same principle each time:
- In the dev loop, Opus reviews Fable's draft with instructions to break it, not bless it.
- In CI, every pull request gets reviewed by a local model running on a Mac in my house — no cloud, no per-token bill — before it can merge. It fails open on a timeout, because a slow reviewer shouldn't block a good change, but it runs on every diff.
- In the 105-agent audit I pointed at my own infrastructure, no finding counted until it survived a separate adversarial verifier. Most didn't. The ones that did were real — including my own exposed API key.
Find, then try to break the finding. It's the same move at every scale.
The pipeline has no taste. I supply that.
None of this is autonomous, and that's the point I'd least want lost. The pipeline generates and it verifies, but it doesn't decide. It doesn't know which of ten possible things is worth building, or which "defect" is actually the intended behavior, or when a passing test is testing the wrong thing. I do. I aim it, I read the verdicts, and I own the merge button.
The leverage isn't that the AI is clean. It's that generation and verification are both cheap now, so I can afford to be ruthless about the second one — and spend my own scarce attention on the one thing neither model has, which is judgment about what's worth shipping in the first place.
Stop grading the first draft. Grade the pipeline. Mine assumes its own author is wrong, and it's right often enough that when it finally says ship it, I believe it.