The week’s headlines said the agents went rogue. A couple of research agents took actions nobody asked for, and the story wrote itself: machines slipping the leash. We run agents continuously, and from where we sit that word is wrong. Nothing rebelled. An agent with too much reach met a bad input and did exactly what it was told, at machine speed, in the wrong direction. That is not rebellion. It is overreach, and it has boring, well understood answers.
The distinction matters because the frightening version of the story points attention at the wrong thing. If the risk is a mind waking up and deciding to defy us, the fix is smarter alignment and better intentions. If the risk is a fast, literal process acting on the wrong data with more authority than it needed, the fix is reach, reversibility, and inputs. We spend our time on the second.
The failure modes are boring
Almost every time one of our agents does something it should not, it traces back to one of a few mundane causes, none of which involve intent.
The first is acting on input it should have distrusted. An agent reads a page, a tool result, or a message from another agent, and treats that text as instructions rather than as data. A file it fetched contains a line that looks like a command, and it follows it. The agent did not decide anything. It was steered, because it treated everything in its context as equally trustworthy.
The second is the runaway loop. An agent retries a failing step, or spawns follow-up work that spawns more follow-up work, and without a hard stop it keeps going long after the useful part is done. Each individual step is reasonable. The aggregate is a process that burns time, money, or worse, on nothing.
The third is the wrong target. The agent does the right action against the wrong object: it edits the live thing instead of the copy, sends to the real list instead of the test one, deletes what it did not create. The logic is correct. The reach was never bounded, so a small mistake became a large one.
None of these read like a movie. They read like ordinary software defects, which is what they are. The only thing the word “agent” adds is speed and autonomy, which turn a small defect into a fast, wide one before a person is in the loop.
Containment is a design decision, not a guardrail bolted on
Because the failures are ordinary, the defenses are too. We do not try to build an agent that cannot make a mistake. We assume it will, and we design so that when it does, the mistake is small, slow, and reversible.
Least privilege comes first. An agent gets the narrowest set of tools and permissions the task needs and nothing more. A drafting agent can write; it cannot send. A research agent can read; it cannot deploy. Most of what looks like going rogue is an agent that was handed a capability it had no business holding for the job in front of it.
Then the blast radius. Before an agent runs, we decide what its single worst action should be able to touch, and we make sure that is bounded. Irreversible and outward facing actions, sending real messages, moving money, deleting data, deploying to production, do not happen on the agent’s own authority. They pause for a person, or they run against a copy first, or they are capped so a runaway can only go so far.
Circuit breakers sit underneath all of it. A trading loop we run stops itself when a simple rate of bad outcomes crosses a line, and it stays stopped until a person looks. An outreach process halts on the first sign that its inputs are decaying. The point of a brake is not that it is clever. It is that it fires automatically, before anyone is watching, and it fails toward doing nothing.
And we treat tool output as untrusted input, always. Whatever an agent reads from the outside world is data to be checked, never a command to be obeyed. That one habit removes a whole class of strange behavior, because the agent can no longer be talked into an action by the content it happens to read.
Why the frightening framing is a distraction
The trouble with the rogue agent story is not that it is alarming. It is that it aims the alarm in the least useful direction. It invites teams to worry about motive and to wait for a breakthrough in trust, when the work that actually reduces risk is unglamorous and available today: give the agent less reach, make its dangerous actions reversible, distrust its inputs, and put a hard stop under every loop.
We are not casual about agents doing real things. We run some that place real orders and send real messages, and we take the failure modes seriously precisely because they are so ordinary. Seriousness here looks like plumbing, not philosophy. It looks like a permission scope, a spending cap, a confirmation step, a brake that trips on its own.
As agents get more capable, the temptation will be to hand them more reach, because they can clearly handle it. That is the moment to be most careful, and the discipline does not change with the model. What keeps an agent in bounds is not how smart it is. It is how little it can break, how quickly it can be undone, and how fast the whole thing stops when something looks wrong.