The Bottleneck in the AI Coding Loop

Benjamin Font Pera described the coding loop as the innermost circle in Loop Engineering, Harness Engineering, Context Engineering: what's the difference?. The agent generates code, verifies it and adjusts it. This round repeats until it considers its result done.

The local build in our last project took eight minutes, including the full test suite. The agents invoked the build in every round. Given how often that happened in a session, the build times added up noticeably.

Writing code was the cheap part in this project. Verification set the pace. How long it takes and how reliable its result is decides how much comes out of a session.

The coding loop is only one link in a longer chain. Behind it lie review, approval and deployment, in front of it coordination and prioritisation. Kai Lichtenberg described what happens when you only turn this one link in From Coding Loop to Business Loop: Thinking AI Engineering Holistically:

Anyone who only accelerates the coding loop optimises locally and loses globally, because value emerges at the slowest link in the chain, not the fastest.

Anyone who halves the build while a finished change waits three weeks for approval has made a micro-optimisation. In our case the bottleneck really was in the inner loop, because the agents ran so many rounds that the eight minutes came up dozens of times a day.

A Slow Sensor Slows Down the Coding Loop

Around the loop lies the harness, which consists of guides and sensors. Guides act before generation and give direction. Sensors act afterwards and report back whether the result holds up. Build, compiler and tests are sensors.

A slow sensor slows down the coding loop, because the agent has to wait for the result before the next step. Armin Ronacher considers this the core of his speed argument in the Agentic Coding Recommendations:

There is a big difference in the quality and speed of the flow, if that tool takes 3ms to run vs it compiles for 5 seconds and then needs another minute to boot and connect to database and kafka broker and 100 lines of nonsensical log output.

Ronacher names Go as an example, because test caching there makes sure that unchanged packages do not run again in the first place. The agent gets the signal in milliseconds and carries on.

The bottleneck in the coding loop is therefore the slowest sensor the agent needs in every round. For us that was the test suite.

If the full check is too expensive to run on every intermediate state, it migrates out of the inner loop, into the pipeline after the commit or into a nightly run. The agent then works with a weaker signal or none at all. Anthropic describes in the Claude Code best practices what happens in that case:

Claude stops when the work looks done. Without a check it can run, "looks done" is the only signal available, and you become the verification loop.

The agent stops as soon as the work looks done. Whether it is done only becomes apparent to the human who reads the code.

Back Pressure

Geoffrey Huntley commented on a text by Moss about back pressure and coined an image that has stuck with me ever since. Back pressure is everything that rejects an invalid generation, so compilers, tests, linters, static analysis, security scanners. His point concerns the dosage:

Back-pressure is part art, part engineering [...] you need "just enough" to reject invalid generations (aka "hallucinations") but if the wheel spins too slow ("tests take a long time to run or for the application to compile") then it's too much resistance.

The wheel in this image turns against a resistance. This resistance can be set wrong in both directions. With too little resistance, hallucinations slip through. Too much resistance brings the wheel to a standstill. A long test suite belongs in this second category. Teams have known the tools for years. If they are missing from the coding loop, the human takes over their work.

Moss argues in his original text about exactly that, namely about the human's time. He starts from the extreme case. An agent that may only write files and can neither compile nor test has to ask the human whether its change holds:

This means you spend your back pressure (the time you spend giving feedback to agents) on typing a message telling the agent it missed an import. This scales poorly and limits you to working on simple problems.

Anyone who spends their time reporting import errors to an agent is giving feedback that compilers and linters would deliver in milliseconds.

Fast Is Not Enough

Verification has two properties, and both have to hold. How fast does the signal come back? How reliable is it? Huntley writes this down himself in his text about the Ralph Wiggum loop:

It's the speed of the wheel turning that matters, balanced against the axis of correctness.

A fast check that verifies nothing accelerates the building of junk. It is more dangerous than a slow, meaningful check, because it fakes trust. AI output looks better than it is anyway. Code can compile and have green tests and still do the wrong thing functionally. Daniel Toews described the pattern where the agent weakens its own tests to go green in Agentic Engineering: Where Loops Fail in Practice and Why.

Verification has to be meaningful first, then it is worth making it fast. In the reverse order you get a loop that delivers a green result in record time that means nothing.

What Can Be Fast and What Cannot

Birgitta Boeckeler distinguishes computational and inferential in her article on Harness Engineering. Computational means deterministic checks that run on the CPU and answer in milliseconds to seconds. Inferential means semantic assessment, for instance by a model, slower and not deterministic. For the inner loop this axis is a usable sorting:

• Milliseconds to seconds: LSP feedback, formatters, linters, compilers and type checkers, ArchUnit rules, targeted unit tests on the affected unit, pre-commit hooks.
• Seconds to minutes: the full unit suite, coverage gates, security scans, static analysis, integration tests with a narrow scope.
• Minutes and more: browser feedback, LLM-as-a-judge, E2E tests, mutation testing, manual review.

The third group does not belong in every build. Boeckeler sums this up as "keep quality left" and calls for checks as far left on the path to production as possible. An error that shows up in the compiler costs the agent three seconds. The same error in an E2E test costs it ten minutes. The test pyramid sorts by the same criterion, because a test takes longer the more application it has to boot up. In our project the weight of the suite was at the top.

From Eight Minutes to Under Two

The local build with the full test suite ran at around eight minutes. The suite contained many tests that had more the scope of an E2E test. These tests have their value, only too many of them ran along in every build. They now run at night.

Before that, the unit tests underneath had to become denser. Where they had gaps, mutation testing showed them, meaning errors deliberately built into the code where it becomes visible whether a test reacts to them. The run takes too long to put it into the build. We started it by hand a few times during the rebuild.

The rebuild moved the tests the agent works on from E2E scope to unit scope. The agent can now run a unit test specifically for the class it is working on, and gets an answer in seconds. A test that needs the full stack always forces it into the full run.

When the Sensor Is a Human

The slowest verification in every project is the human. Addy Osmani describes in Agentic Code Review:

writing code was the slow, expensive part, and reading it was cheap and fast. That fact no longer holds. [...] the constraint moved downstream, to the one step that did not get faster.

Every sensor in the coding loop could be made faster in our project, so compilers, tests and linters. For reading a pull request there is no such lever. Anyone reviewing a thousand changed lines needs as long as they always have.

A faster coding loop pushes this further. The agent manages more rounds, so more changes come into being that someone has to read. What the build finds itself never reaches the reviewer in the first place.

Huntley made a remark about pre-commit hooks that picks this up. Hooks were always considered annoying because they hold humans up. When the agent makes the commit, the agent waits for the hook. Nobody pays those seconds out of their own time any more. An error the hook catches no longer arrives in review.

Where Humans Remain

The human thereby moves from the inner to the outer loop. Osmani calls this "human on the loop". That matches what we consider the target model in our own projects. We build the frame, watch what slips through, and improve the frame.

One question remains the human's responsibility. Automation structurally does not help there. Simon Willison put it this way in Hallucinations in code:

No amount of meticulous code review—or even comprehensive automated tests—will demonstrably prove that code actually does the right thing.

Whether the code runs is settled by the build. For the question of whether it was the right change, there is no sensor. On top of that come the paths where an error radiates far, so security, architecture, everything with a long lifespan. Human reading is still worth it there. The effort is decided by the risk of the change. The question of who wrote it plays no part. Whoever lets a change into the main branch carries the responsibility for the result, even if a review agent approved it beforehand.

The trivial feedback belongs in automation, so duplicate imports, formatting, obvious regressions. What remains is the question of whether the result is functionally correct. That needs humans who still have capacity.

The same question comes up on every level above the coding loop. How long does it take until feedback is there? How much of it runs on its own? A stage can only be automated if a tool takes over the feedback. In the coding loop these tools have been available for years. On the levels above you usually have to build them first and lay out the process for them.

Agents Act Like a Magnifying Glass

The eight minutes were just as long before. Between two builds there used to be half an hour of thinking and typing, though. The waiting time disappeared in that. The pace of the agents acts on such numbers like a magnifying glass. What used to hide in the rhythm of human work steps is now in the way dozens of times per session.

Under this magnifying glass lie the things that annoyed teams before the agents too. A slow build, a test suite with too much E2E weight, unit tests with gaps, a review backlog before the merge. None of this is new. Recognising these problems did not require AI before either. What is new is how much they cost now.