Blog
AI coding agents are moving the bottleneck to verification
· agents, engineering, evaluation, analysis
What recent research says about agent productivity, why more output can slow delivery, and a practical evaluation worksheet for small engineering teams.
A team can generate working-looking code faster than it can decide whether that code deserves to ship. As coding agents become more capable, that gap becomes a management problem as much as a model-selection problem.
The useful question is how quickly a team can turn a proposed change into a result it trusts. A faster first draft helps. It does not tell us how long someone will spend reproducing an issue, reviewing the change, checking an assumption or recovering from a bad deployment.
Recent research gives this distinction sharper edges. This article reads three primary sources, then proposes a small-team experiment. The experiment is our recommendation, not a benchmark we have run or a claim about TopxAI's performance.
What the new evidence actually establishes
In its September 6 research update, OpenAI reports growing use of coding agents inside its research organization, alongside more code and experiments. It also gives an unusually useful qualification: over half of successful tasks estimated to take a human four to eight hours involved at least one intervention. Its task-success analysis excludes uncertain outcomes. More available compute is another factor in the experiment-volume trend.
These are observations from a frontier lab, with its own infrastructure and task mix. They do not establish the same productivity gain for a five-person SaaS team. The report itself cautions that activity metrics are difficult to translate into overall research progress. That limitation deserves to travel with the headline.
My interpretation is that delegation is becoming useful at a wider range of task sizes while supervision remains part of the production process. Counting the delegation and omitting the supervision would describe only part of the work.
Two earlier engineering reports help explain what to measure. In February, Anthropic varied infrastructure resources while holding the model, harness and tasks constant on Terminal-Bench 2.0. It reported a six-percentage-poi