Back to News & Insights
Artificial Intelligence September 9, 2026 · 8 min read

The Verification Bottleneck in AI-Generated Software

AI can generate code faster than ever. That doesn't mean we're shipping correct software...

The Verification Bottleneck in AI-Generated Software

AI can generate code faster than ever. That doesn't mean we're shipping correct software faster.

Recently, I asked a coding agent to build a password reset flow. It produced the route, the token handling, the email integration, and the UI in about five minutes.

The implementation had a bug. The reset link worked more than once. Use it to set a new password, then open the same link again, and it still worked.

The feature request hadn't explicitly said that a reset link should be single use. It wasn't in the feature request, because it's the kind of thing that goes without saying right up until the moment it doesn't. Every path a person would click through by hand worked perfectly. A code review might have caught it. A demo would not have.

Traditional software development has an obvious constraint: humans have to write the software. We think about the requirement, design the implementation, write the code, run it, discover that it doesn't work, debug it, and repeat until we're sufficiently convinced.

AI coding agents compress parts of that loop dramatically. The agent can often produce an implementation faster than I can thoroughly review what it generated, which creates an inversion. For a long time, writing code was expensive and checking it was comparatively cheap. What happens when writing becomes cheap?

Suppose an agent implements a feature in five minutes, but determining whether the implementation is correct requires another 45 minutes of manual testing and code review. We haven't created a five-minute development process. We've created a 50-minute development process with a very fast implementation stage.

And the verification half is harder than it used to be, because you're now auditing code you didn't write.

If an agent can rewrite a component cheaply, we become less attached to any particular implementation. Implementations are becoming disposable. Six months from now the routes change, the framework changes, the DOM changes, the internal architecture changes.

A user who resets their password must subsequently be able to authenticate with the new password, must no longer be able to authenticate with the old one, and must not be able to reuse the reset link.

In an era of inexpensive code generation, the thing worth maintaining is increasingly the specification rather than the implementation. Which means it's worth writing the specification first, deliberately, as an artifact in its own right rather than as documentation of something already built.

The application was deliberately ordinary: authentication, user accounts, login behavior, the kind of functionality in countless business applications. I wasn't interested in whether an agent could produce something visually impressive. I wanted to answer a different question. Can we define expected behavior first, let an agent implement it, and use independent end to end verification to determine whether the agent actually succeeded?

Consider the password reset feature. The implementation involves a route, form handling, token generation, validation, password hashing, database updates, session behavior, error handling, and UI changes. An agent generates all of it quickly.

None of those implementation details are what the user cares about. The user cares whether they can request a reset, receive the email, set a new password, log in with it, and whether the old password and the used link both stop working.

Those are observable outcomes. That makes them a useful boundary between what we asked the agent to build and what the agent actually built.

testRigor expresses tests as English-like behavioral instructions rather than requiring the test author to work primarily in selectors and automation code. A test describes what a user does and what they expect to see.

That creates an opportunity when combined with AI-assisted development, because the behavioral test can function as an executable specification.

Instead of telling an agent to "add password reset," we provide a behavioral contract. The application must email a reset link. The link must lead to a form. The new password must work afterward. The link must not work a second time.

Want to discuss this further?

Book a free strategy call with our team to see how these insights apply to your specific business goals.

Book a consultation