Researchers Decoded 315,320 'Encrypted' AI Reasoning Blocks and Pulled Out 62 API Keys, 33 Passwords and 7 Private Keys. The Encryption Was Never Protecting You From That.
- Patrick Duggan
- 23 hours ago
- 4 min read
A paper published today — Stealing Reasoning Traces from Proprietary LLM APIs, arXiv:2608.09867, from a team spanning the ELLIS Institute Tübingen, the Max Planck Institute, MATS Research and Snyk — does something more useful than demonstrate a clever attack. It measures the damage on real data.
The researchers took 6,708 publicly published agent trajectories, decoded 315,320 encrypted thinking blocks, and recovered 704 distinct privacy artifacts. Among them: 62 API keys, 33 passwords, 24 access tokens, and 7 private keys.
Those are not lab findings. Those are live credentials that people published, believing the reasoning blocks in their agent logs were opaque.
How it works, and why "encrypted" was the wrong word to hear
Frontier providers return their models' internal reasoning to API callers as an encrypted block. You get an opaque object; you pass it back on the next turn so the model keeps its train of thought across a stateless conversation.
The attack is not cryptographic. Nobody broke the encryption.
The reasoning blocks turn out to be portable — across sessions, across users, and, critically, across models within the same provider family. So you take an encrypted block produced by a frontier model, hand it to a weaker sibling in the same family, and ask that sibling to transcribe it. The weaker model can read the block, because it is family-compatible, and it has thinner guardrails, because it is not the model anyone hardened against this.
The decoders named in the paper are specific: Claude Haiku 4.5 for Claude traces, GPT-5.6 Luna for GPT traces, and Gemini Robotics ER-1.6 for Gemini traces. Standard, unprivileged API access throughout. No exploit, no privilege escalation, no vulnerability in the conventional sense.
Here is the thing worth internalizing: that encryption was doing its intended job. It exists to stop competitors distilling frontier reasoning into their own models — an anti-distillation control, and a reasonable one. It was never designed as a confidentiality boundary protecting the contents of a trace from whoever is holding the trace.
The word "encrypted" did the rest. Engineers saw an opaque blob, concluded it was safe, and logged it, shipped it in bug reports, and pushed it to public repositories. The control worked. The inference people drew from it did not.
Your observability stack is now a credential store
That is the operational finding, and it is the one to act on today.
If you run agents, your LLM logging, tracing and evaluation tooling is almost certainly retaining reasoning blocks. LangSmith, Braintrust, Phoenix, W&B Weave, homegrown JSONL in S3, the debug output attached to a Jira ticket — all of it. Those retained blocks can contain any secret the model reasoned about, in plaintext, once passed through a sibling model.
And it is worse in the places you would least suspect, because the 6,708 trajectories in this study were public. Agent trace dumps get published constantly and casually: in eval harness repositories, in reproduction cases filed against open-source frameworks, in blog posts demonstrating a cool agent run, in benchmark artifacts. Every one of those is a potential key disclosure that no secret scanner is looking for, because no secret scanner knows how to decode a provider's reasoning envelope.
Three things worth doing:
Treat reasoning blocks as secret material at rest. Same handling as the prompt and the tool outputs. If your retention policy says "we keep traces for 90 days for debugging," you have a 90-day credential store.
Strip reasoning blocks before anything leaves the building — bug reports, support tickets, public repos, benchmark submissions. This is a five-line filter and nobody has written it because nobody thought it was necessary.
Stop putting secrets where the model can reason about them. The durable fix is the boring one: credentials belong in the tool-execution layer, injected at call time on the server side, never in a context window. If the model never sees the key, no trace can leak it.
The three providers, reported flat
Their responses differ, and we are going to state that plainly in both directions.
Anthropic guidance is the strictest of the three: thinking blocks are tied to the model that produced them and should be stripped when switching models.
Google now handles thought compatibility in the backend when a session switches models.
OpenAI still directs developers to replay encrypted items for stateless history management — the pattern the attack depends on.
Disclosure, because it matters here: we build on Claude, and Anthropic is our platform partner. On this particular finding the guidance that reads best happens to be theirs. We are reporting it because it is what the paper says, and we would report the inverse the same way — we have published Anthropic-affecting findings before and will again. If you think this reads soft, the paper is public, the arXiv number is above, and you should check it rather than take our word.
The honest caveat cuts the other way too: the researchers' own reproducibility statement says the main extraction attack is no longer reproducible as of August 2026, though no formal vendor confirmation has surfaced publicly. So the live window may be closed. The 704 credentials already sitting in already-published traces are not un-published by that, and nothing about a backend fix retroactively scrubs the logs you have been keeping since last year.
What we had
Nothing. This is academic research disclosed today, there are no indicators to hold, and there is nothing in our feed that would have caught it. Credit is entirely to the paper's authors, with a nod to Matthew Green's prior work on replay behaviour in these APIs, which the paper builds on.
Our contribution is the framing: this is not an AI safety story, it is a secrets management story wearing an AI costume. The failure mode — an opaque encoding mistaken for a security boundary, then logged everywhere — is one the industry has hit before with JWTs, with base64, with session cookies. It will keep happening as long as "I cannot read it" gets treated as "nobody can."
Ninety-five percent. If your agent framework strips these blocks by default, tell us and we will name it, because that is a control worth pointing people at.
How do AI models see YOUR brand?
AIPM has audited 250+ domains. 15 seconds. Free while still in beta.
Was this useful? Thirty seconds, no cookies, no tracking, no third parties, your address hashed and never stored. If the box below does not load, the same question lives at https://analytics.dugganusa.com/nps.html?post=researchers-decoded-315-320-encrypted-ai-reasoning-blocks-and-pulled-out-62-api-keys-33-passwords
Comments