AI can generate the code. We can verify the behavior. But who decides what correct means?
I wrote recently about a coding agent that built me a password reset flow with a reset link that worked more than once.
The bug survived because nobody had written down that a reset link should be single use. It was obvious right up until it wasn't.
My argument was that as AI makes implementation cheaper, verification becomes the bottleneck. The feature request said "build password reset." The agent built password reset. The happy path worked. The tests passed. The implementation looked finished.
So I added an independently written behavioral specification. The agent implemented against it. The verifier rejected the reusable token. The agent fixed the implementation. The verifier passed it.
Then I published the article, and the comments started finding things my specification didn't say. That exposed a harder problem.
One reader asked what would happen if two password reset requests using the same token arrived at the same time.
Both requests check the token while it is still unused. Both proceed. If validation and consumption are not a single atomic operation, "single use" can still produce two successful resets.
That doesn't make the specification useless. It makes the specification provisional.
Once somebody discovers that "single use" also means competing attempts cannot both succeed, that should stop being knowledge held by the person who noticed it. It belongs in the durable definition of correct behavior.
Another reader described an integration builder where an agent wrote both a connector and the tests for that connector.
The connector and its tests encoded the same incorrect assumption about OAuth token refresh. The mistake only surfaced when a customer's token expired during a live session.
The implementation and test suite were separate artifacts. They were not independent in the way that mattered.
That distinction matters because "independent verification" can sound like an organizational property:
different file different test suite different agent different step in the pipeline
If the implementation and verifier derive their definition of correct behavior from the same incomplete prompt, they can agree perfectly and still be wrong.
Another commenter asked whether property-based testing or giving an agent an adversarial security persona might do a better job uncovering these unstated constraints.
I think both are interesting, but they expose the same boundary. Property-based testing can explore a stated invariant extremely well.