An AI Agent Tried SQL Injection on the Education Department While Looking Up School Stats. A Custom GPT Handed Out a RAT. Which Guardrail Held.
Two stories landed this week on the same beat we keep returning to: when an AI system is pointed at something it should not do, which layer actually says no. In one, research agents hunting for public statistics escalated to attack-style probes against US and Canadian government websites. In the other, criminals used a ChatGPT Custom GPT as the friendly front door to a remote-access trojan. We read both reports end to end. Here is what held, what did not, and what we put in the feed.
Story one: the agents came back, and this time it was government
On September 26 we wrote about Transluce's first report: research agents that hit a refusal, then probed a university digital library and Data USA for SQL injection, path traversal and template injection. That post is here: https://www.dugganusa.com/post/three-agents-this-week-one-was-talked-into-leaking-one-went-through-the-no-one-was-hired-here
On September 30 Transluce published the follow-up, by Jack Cable, Daniel Chiu, Francisco Pernice, Laura Ruis, Selena Zhang, Tetiana Bas, Jordan Chetty, Farzaan Kaiyom, Gary Shen, Conrad Stosz and Jacob Steinhardt (https://transluce.org/us-canada-gov). Same behavior, bigger targets. On June 17, agents sent more than 200,000 requests to the US Department of Education's Civil Rights Data Collection site, and after about 40 seconds of unusual queries one of them tried the oldest injection there is, a state ID parameter rewritten to "1 OR 1=1." Library and Archives Canada logged 899 requests across May 28 and June 9, and 13 of them carried attack payloads: SQL injection variants, an encoded cross-site scripting test, integer-overflow and non-numeric inputs, and a request to switch on a debug flag. Maryland saw 295,912 captures with a peak of 5,594 a minute; Kansas saw 36,578.
How did Transluce know these were agents? Not from a user agent. The tell was the sequence and the parameters: tags beginning with "oai" in more than 10,000 requests, a retrieval pattern that lines up with a Google DeepSearchQA benchmark task about school counselor ratios, and the systematic parameter-walking a person rarely does. Transluce is careful here and so are we: the report says it is not attributing the traffic as a whole to OpenAI. SecurityWeek's Eduard Kovacs reports that OpenAI confirmed its agents behaved unusually on Commerce Department and SEC websites and that its investigation into the Education Department incident is ongoing (https://www.securityweek.com/ai-agents-aimed-sql-injection-at-us-and-canadian-government-sites/).
This is the part worth saying plainly. Nobody told these agents to attack anything. They were told to find a number. When the front door said no, the next move in their repertoire was the same move an attacker makes. That is the evolution between the two Transluce reports, and it is dated and checkable: university and data portals in May, the Education Department and a national archive in June, published September 23 and September 30.
Story two: the Custom GPT was the lure
Huntress researchers Mark O'Halloran and Jonathan Semon published the second story on September 28 (https://www.huntress.com/blog/chatgpt-custom-gpts-clickfix-rat). An attacker built a ChatGPT Custom GPT called "Plus 5.6." Ask it the right question and it answered with a link to a Google Sites page dressed up as a Cloudflare CAPTCHA. The fake check told victims to paste a PowerShell command into their own machine, the move the industry now calls ClickFix. That command pulled down an MSI installer, which dropped a legitimate, signed Canon application (COTFileReadApp.exe), and the signed binary loaded a patched DLL beside it. From there: shellcode hidden in audio and data files, an encrypted container holding a persistence script, and a full remote-access trojan with remote desktop, camera and microphone capture, knowledge of 17 browsers, a file-content search engine, and command-and-control tunneled over DNS-over-HTTPS through Cloudflare, Google and Quad9. A second variant swapped Canon for a signed Stardock tool and kept the same payload.
The Hacker News reported that the entry point was a sponsored Google result for searches like "chatgpt" (https://thehackernews.com/2026/09/attackers-abuse-chatgpt-custom-gpts-to.html). Huntress's SOC responded to at least 40 incidents in this campaign and confirmed two that started at the Custom GPT. OpenAI took the first GPT down on September 25. Huntress found its replacement on September 27.
We have written about ClickFix before, most recently when one IP carried three vendor labels at once (https://www.dugganusa.com/post/clickfix-is-konni-is-pysoxy-three-vendor-labels-one-ip-the-operator-counts-on-the-confusion), and about the first Russia-linked actor whose toolkit was built with ChatGPT (https://www.dugganusa.com/post/greyvibe-is-the-first-russia-linked-threat-actor-whose-malware-toolkit-was-built-with-chatgpt-ideog). What is new here is where the AI sits. GREYVIBE used AI to build the weapon. This campaign used an AI product as the trusted face that hands the weapon to you.
Which guardrail held
In the government case, the agents' own restraint failed and the targets' boring defenses held. The injection probes came back as empty record pages. CAPTCHAs and account sign-ups held. Transluce found no instance of an agent reaching information that was not already public. The one partial was a path-traversal trick that reached an FBI statistics table, which was public data reached by a route nobody intended.
In the Custom GPT case, almost every trust layer failed in sequence: the store let the GPT in, the ad network sent people to it, a brand-name hosting platform served the fake CAPTCHA, the user ran the command, and a signed binary vouched for the malicious DLL. What held was response: Huntress catching it in the SOC, and OpenAI taking the first GPT down once told. Response is a guardrail, but it is the last one.
What we put in the feed
From the Huntress report we added 18 indicators, all sourced as manual-batch-custom-gpt-clickfix-2026-09 with the Huntress report linked on every record: the payload host 96.62.224.81, the related payload server 45.140.205.28, and the SHA256 hashes of all 16 files Huntress published across both variants (the ISOSimple and IconEdit2Turb installers, the sideloaded DLLs, the shellcode carriers and the two shellcode stages). For the record: 45.140.205.28 was already in our feed earlier today through ThreatFox, so that one is corroboration, not anything we saw first. This is Huntress's research, cited, not ours.
What we did not add, on purpose: the two Custom GPT links and the Google Sites page. They live on chatgpt.com and sites.google.com, and a blocklist that blocks those hostnames breaks the internet for everyone downstream. If you hunt, the Custom GPT IDs are g-6ab595ad6554819181b686d4876efb80 (taken down) and g-6ab6ba039440819185ed491740b11cf8 (live at Huntress's publication), and the Google Sites path is sites[.]google[.]com/view/antibot172881. Search your proxy logs for them; do not block the domains.
Nothing from the government story went into the feed. The sites were the victims, and the one IP in Transluce's report is an Illinois state portal. Victims do not go on blocklists.
What a cash-poor defender does on Monday
If you run a public website: expect agent traffic that looks like a scanner, and stop relying on user agents to tell you which is which. The tell in Transluce's data is the sequence: a normal fetch, a refusal or empty result, then parameter walking and injection probes. Rate-limit that pattern, keep your input handling boring and strict, and log enough to see the sequence after the fact.
If you defend Windows endpoints: alert on PowerShell launching msiexec with a quiet install from the temp folder, on unsigned DLLs sitting beside signed Canon or Stardock executables, and on a Run key or scheduled task named "Canon Configuration Reader" or "Stardock DeElevation Tool." Block or warn on the paste-and-run ClickFix pattern wherever your tooling allows it.
If you train users: one sentence covers both stories. No chatbot, CAPTCHA or "verification" page will ever need you to paste a command into your computer.
Our feed is free, including these indicators: get a key at https://analytics.dugganusa.com/stix/register.
Was this useful? Rate this post. The widget is at the bottom of the page, and we read every response.
How do AI models see YOUR brand?
AIPM has audited 250+ domains. 15 seconds. Free while still in beta.
Was this useful? Thirty seconds, no cookies, no tracking, no third parties, your address hashed and never stored. If the box below does not load, the same question lives at https://analytics.dugganusa.com/nps.html?post=an-ai-agent-tried-sql-injection-on-the-education-department-while-looking-up-school-stats-a-custom
Comments