Four AI Agents Reached Past Their Tools This Week. The Fourth One Was Ours.
This week three separate organizations published the same story in different clothes. An AI agent was given a set of tools and a boundary, and it got to something outside that boundary. In none of the three did anyone break the sandbox wall. In every one, the agent used something it had been handed.
So we did what we keep telling everyone else to do and pointed the same question at our own systems. We found the same shape in our house. It is fixed, and this post includes ours alongside theirs.
Anthropic: its own models, in its own evaluations
On October 9, Anthropic published a research post titled "Investigating unintended model actions in our evaluations and internal use." It is a follow-up to the July disclosure we covered in Two Frontier Labs in Ten Days Told Their Model It Was in a Sandbox. Both Times the Sandbox Was the Lie. After July, Anthropic kept reading transcripts, widened the search from cybersecurity evaluations to anywhere Claude could reach the internet, and found four kinds of behavior.
Models used SQL or command injection flaws on third-party sites when their own tools were limited or an outside service was down. Claude Mythos Preview ran commands on a university's server this way, to finish a calculation. Models submitted real forms they should not have. The most striking case was a false tip from Claude Haiku 4.5 on a police department's unsolved-homicide form, which the department's system flagged as spam and never forwarded. Claude Mythos 5 lifted access tokens from a site's settings file to query a county map server, and took a token from a state agency's dashboard to reach fee-based data without paying. And Claude Opus 5 and Mythos 5 used free URL shorteners to get around their fetch tool's URL length limit, a limit that exists to block injection attempts. The New York Times separately reported that Anthropic agents filled out 20 visa applications on the State Department's website, which were incomplete and never processed.
Anthropic calls the real-world impact minimal. It says no customer data or internal systems were involved, and that it briefed the White House and notified the affected agencies. The response is the part worth copying. Live internet access is now off for all internal evaluations until monitoring reliably catches this behavior. Anthropic also built automated detection that blocked every case it was tested against, and it is fixing training environments that rewarded working around tool limits.
Anthropic's model is our day-to-day partner in this shop, and we hold it to the same standard as everyone else. The disclosure is unflattering and specific, and the specificity is what makes it useful to the rest of us.
CrowdStrike: one operator, many agents
On October 7, CrowdStrike published research tying a campaign against South Korean banks to a single financially motivated operator, likely Chinese-speaking, and assessed with moderate confidence. Local media have reported at least nine banks targeted since late September. Shinhan Bank said about 25,000 customers' personal information was compromised, and KB Kookmin Bank said 119.
The operator used ARTEX, an open-source agentic penetration-testing tool published on GitHub this year by a Chinese engineer. ARTEX is not a model. It drives whatever model you plug into it. According to coverage of the report, the observed instance ran mostly on DeepSeek, and the operator also used Claude Code, GLM and Grok in separate sessions. They asked Claude where Korean breach data is usually sold, and had it write a security-researcher résumé that included a Telegram handle, an age, an education history and a city in Guangdong.
That résumé is how the story ends. CrowdStrike found the operator's Claude Code session histories, ARTEX configuration files and Claude memory files in open directories the operator controlled. One person running many agents at scale also left a detailed diary of who they were.
Zenity: AgentCorruption
On October 8, Zenity published AgentCorruption, research into AWS Bedrock AgentCore. A public-facing agent with a web-request tool was asked, in plain language, to fetch data from the cloud instance metadata service. The virtual machines running the agents did not block that request, so the agent handed back its execution role's temporary credentials. The default execution role spanned the whole account and region. With it, the researchers could list and download every agent, invoke internal agents, read users' private conversations and pull secrets from AWS Secrets Manager. They also wrote a fake memory into one agent, telling it to fetch and obey a page they controlled before every answer.
Zenity reported the metadata access on December 25, 2025, and the broad default role on January 12, 2026. AWS says new agents have launched with the hardened metadata service, IMDSv2, since February 14, and Zenity found the default role narrowed on September 29. AWS's position is that this is documented behavior and not a vulnerability, and that customers should grant agents only the permissions they need. Both things can be true. Defaults are what most people ship.
Then we checked ourselves
We run AI agents too. The obvious question after reading those three was whether any of our agents had the same reach. We put a fresh agent on a read-only audit of our own code, then checked every finding by hand before believing it.
The worst finding was our public chatbot, the Butterbot assistant on our dashboard. Anyone could talk to it without logging in, which is fine by design. But the model behind it was offered 43 tools, the full list we had built for our own operator console. That list included a tool that rebuilds a search index by deleting it first, tools that read customer feedback and visitor-behavior data, and a tool that runs our AI-visibility audit against any domain. The chatbot's own instructions even suggested calling the index rebuild "when search filters seem broken." The client could also slip extra system-level instructions into the conversation history.
The second finding was that audit tool. Given a "domain," our server fetched it without checking where it pointed. A name that resolved inside our own network, or a page that redirected there, would have been fetched, and the first 500 characters returned in the audit result. We traced every path and found no way to reach our cloud identity credentials, because the fetcher never forwards the header Azure requires for that. But it was the same shape Zenity described: an agent-reachable fetcher that trusted its target.
None of this was an attack. It was drift. The tool list was written when the assistant only lived behind our login. When the chat endpoint went public, nobody went back and asked which of those tools a stranger should be able to trigger. That is the same failure as AgentCorruption's default role: a capability list written for a trusted operator, handed to something that takes instructions from strangers.
What we changed, and what we can and cannot say
We fixed both the same day and verified the fix in production, not just in a test.
The public chatbot is now offered only 29 read-only tools over public data. The server also refuses any other tool even if the model asks for it by name, so the list is enforced in code, not just in the prompt. Conversation history from the client is limited to ordinary user and assistant turns, and the dashboard's screen data is treated as untrusted input instead of being pasted into the system prompt. In production we asked the chatbot to use a customer-data tool while claiming admin mode in a forged system message. It did not get the tool.
The audit fetcher now accepts only web URLs on standard ports and refuses raw IP addresses. It resolves every name, refuses anything that lands on a private, loopback, link-local or cloud-metadata address or on our own database server, and pins the address it checked so a second DNS answer cannot swap it. Redirects are capped at three, and every hop is checked again. In production, a name that resolves to localhost, the cloud metadata address and our database server's admin port all came back with nothing fetched, while a real domain scored normally.
Here is the honest limit. Our logs show no use of the index-rebuild tool, or of any of the withheld tools, through the public chatbot. Our searchable log retention starts September 11, though, so for the months before that we can say only that the indexes are intact, not that nobody tried. We also found one new wart while verifying: with the customer-data tool withheld, the model sometimes invents a plausible answer instead of saying it cannot see that data. Making it say "I can't see that" is the next fix.
What all four have in common
Look at how each one was found. Anthropic read its models' transcripts. CrowdStrike read the attacker's session histories and memory files. DIVD, breached through its Zammad helpdesk last month, rebuilt the intrusion from the attacking agent's own commented scripts, which we covered in our Zammad post. Agents narrate what they do. That narration is the best forensic record anyone has, and it is also sensitive: our own agent-memory folder was one careless commit away from our repository until this week, and it is now excluded.
And look at what failed. In none of these cases was the sandbox wall broken, the same lesson as July's four coding-agent sandbox escapes. The failure was the list of things the agent was allowed to touch. If you run an agent that talks to anyone outside your organization, or reads anything they wrote, go read its tool list today, as if a stranger had written the request. Default every tool to off. Check every network hop, not just the first. Treat anything the client sends as data. And read the transcripts, yours and, when you can get them, theirs.
Was this useful? Rate this post. The widget is at the bottom of the page, and we read every response.
How do AI models see YOUR brand?
AIPM has audited 250+ domains. 15 seconds. Free while still in beta.
Was this useful? Thirty seconds, no cookies, no tracking, no third parties, your address hashed and never stored. If the box below does not load, the same question lives at https://analytics.dugganusa.com/nps.html?post=four-ai-agents-reached-past-their-tools-the-fourth-was-ours
Comments