The single change that improved our agents the most last quarter was not a new model and not a better prompt. It was a three-second tool call we got down to two hundred milliseconds.
We had been telling ourselves a different story. The way to make the agents better was to wait for a smarter model, or to find a sharper prompt, or to give them more context. We tried all of those. They helped in the way upgrades usually help: visibly for a week, then absorbed. The thing that did not absorb was tool latency. Once we started timing our tools, we kept finding cases where the loop, not the model, was the ceiling.
What follows is three of those cases. Two are tools we changed. One is a tool we deliberately left alone. They look unrelated at first. The common thread came later.
The search tool that changed how we searched
Our file-and-symbol search used to take three to four seconds. Not catastrophic. Slow enough that the agent treated each call as a bet. It would write a search query, weigh whether the search was worth the wait, and most of the time work from whatever came back without trying again.
We rewrote the index. The same query now returns in about two hundred milliseconds. The wall-clock saving was real but small. The interesting change was in how the agent used the tool. It started searching three or four times in a turn where it used to search once. It checked names instead of guessing them. It looked up where a function was defined before editing the call site. The patches it wrote got more accurate, not because the model had improved, but because the model could afford to confirm things it used to assume.
We had not predicted that. We had predicted faster turns. What we got was a different kind of turn, with more steps, each one cheaper and more grounded. The throughput gain was not “same loop, faster,” it was “the agent stopped reasoning at half-resolution because the cost of looking was suddenly low.”
The file read we capped instead of speeding up
The opposite happened with our file-read tool. It was fast enough. The problem was what it returned. A read on a four-thousand-line file dumped four thousand lines into the context. We never sped it up. We capped it.
The tool now requires a line range, defaults to a small window, and exposes head, tail, and next-page calls. The wall-clock latency did not change. The reasoning quality did. The agents stopped scrolling past noise to find the one function they needed. Their windows stayed smaller, which meant later turns had more room to think, which meant the work stayed coherent for longer before we had to reset state.
This was the trade we kept missing. A tool that returns too much output is a slow tool in disguise. It is fast on the clock and slow in the context. The cost is paid by every subsequent decision the model makes against that fatter window, and we do not see the cost in any tool-call timing dashboard. We see it later, when the agent forgets what it was doing and asks a question we answered six turns ago.
After the cap, we stopped measuring our read tool only by latency. We started measuring it by average bytes returned per call. The two numbers belong on the same chart. They are both ways of describing the weight a tool drops into the loop.
The slow tool we left alone
Our build-and-deploy tool takes about ninety seconds. We have not optimized it. We probably will not.
The reason is not principle. It is shape. The agent calls that tool once per ticket, sometimes once per day, not once per turn. It sits outside the loop the agent does its thinking inside. A faster build would not make the agent reason better. It would save us some wall-clock time at the end of a piece of work. That is a real saving, but it is not the same kind of saving as the other two, and we have learned to ration optimization attention by where the call sits.
What we did change was the agent’s behavior around the slow tool. We made it batch. The instruction now is to stage a set of changes, run the build once when the set looks coherent, and only iterate after the build returns. Before that change, the agent would sometimes call build after a single-line edit, wait ninety seconds, get a green, and then make the next single-line edit. That pattern made the slow tool feel like a wall-clock problem. After the change, it does not.
Knowing whether to make a tool faster or to keep agents away from it is its own decision. Both are valid. The question is whether the tool is on the agent’s reasoning loop or at its edges. The same ninety-second cost in two different positions is a very different cost.
What feedback-loop speed actually buys
The easy framing is wall-clock. Faster tools, faster turns, more work per hour. That framing is true but partial. It misses the part that matters most.
Every tool call adds something to the window the model reads next. A slow call makes the agent treat the call as a bet, which lowers how often it checks. A fat call fills the window with material that crowds out the task. A well-placed call returns quickly and returns little, and the room left in the window stays available for the actual reasoning. Latency, output size, and call frequency interact. We had been measuring only the first.
Once we put all three on a chart, the optimization decisions stopped being mysterious. Search was a hot-path tool returning small payloads. Speed mattered most. The read tool was a hot-path tool returning huge payloads. Output size mattered most. The build tool was a cold-path tool. We left it alone and changed agent behavior instead. The same lens decided all three.
The result that surprised us, and that has changed how we think about the layer around the model in general, is that model speed turned out to be the smallest lever we had. The model is fast and getting faster. The tools around it are where we lose minutes per ticket if we are not careful, and where we lose reasoning quality even when wall-clock looks fine.
We log tool latency next to model latency now. New people on the team are usually surprised by the ratio. The model is the cheap part. The tools are most of the loop. Whatever we ship next is going to start there.