We Built a Free Scanner for Secrets Hiding in Claude Code Settings Files. Two Security Reviews Found It Could Be Attacked by the Repos It Scans, and Here Is How We Fixed That.
If your team uses Claude Code, run one command on your code folder today: npx claude-settings-leak-check --root ~/code. It looks for secrets saved in Claude Code's settings files, says how far each one could have travelled, and never prints the secret itself. It is free, MIT licensed, has no dependencies, runs entirely on your machine and makes no network call. The code is at github.com/pduggusa/claude-settings-leak-check.
We built it the day after Apiiro's research on the "Yes, and don't ask again" file, which we covered in "Apiiro Found 1,122 Public Copies of Claude Code's Settings File Holding Secrets." This post is about the tool, what we found when we pointed it at ourselves, and two bugs in it that matter more than the tool does.
What it does
For every .claude/settings.json and .claude/settings.local.json under the folder you point it at, it reads the permission lists and the env block and looks for known secret shapes. Those include GitHub, Anthropic, OpenAI-style, AWS, Stripe and Slack keys, JWTs (which is what a Supabase service-role key is), private-key blocks, passwords inside URLs, bearer and basic auth headers, password flags on command lines, and assignments like NAME_TOKEN=value. It also flags ANTHROPIC_BASE_URL when it points anywhere but Anthropic, because that setting would send your login token to whoever runs the server.
Then it grades the file by how far the secret could have gone. LOCAL means the file is git-ignored and was never committed. AT-RISK means it sits in a git repo and is not ignored. COMMITTED means it is or ever was in history. EXPOSED-REMOTE means committed and the repo has a remote. The output shows the rule, the position in the list and a masked prefix with a length, such as ghp_…(40 chars), so a report is safe to paste into a ticket. Exit code 2 means a finding, which makes it usable as a CI step.
What we found on our own machine
Nothing real. The first run flagged two entries in our local settings file, and both turned out to be the same 15-character placeholder, not a credential. It matched none of the 68 secrets in our vault. So the first bug was in the tool: it did not know a placeholder from a secret. It now skips values like __KEY__, <token>, ${VAR} and REDACTED. Our settings file is git-ignored and has never been in a commit. We scanned nine settings files across our working folders and found no known secret pattern.
That result is worth little on its own, and the tool says so every time: no finding is not proof of clean.
The part that matters more: the scanner could be attacked
We pushed the tool to a public repository and an automated security review ran on the commit. It found a real problem. The tool runs git inside each repo it scans, to learn whether a settings file is tracked or ever committed. A repo's own .git/config can name programs that git will run. One setting, core.fsmonitor, names a command that git runs when it checks a repo. So scanning a hostile clone could run the attacker's command on your machine. A tool for finding leaks would have been a way to get hit.
We fixed it by overriding those settings on every git call, and wrote a test with a hostile repo whose fsmonitor setting tries to create a file. The test fails on the old code and passes on the new.
The second review found a sibling. The tool used git log to check history, and git log can run a program named in gpg.program when log.showSignature is on and a commit carries a signature block. We reproduced it by hand: a marker file appeared. We switched to rev-list, which does not check signatures, overrode the gpg programs and told git to ignore global configuration.
Here is the part we want to be straight about. Our first regression test for the second bug passed against the vulnerable code, so it proved nothing. The reason was a third bug. On a Mac, /var is a link to /private/var, and git reports the real path, so the tool built a nonsense path for the file and quietly graded it wrongly. It now resolves the real path first. With that fixed, the test fails on the vulnerable version and passes on the fixed one, which is the only way we trust a test. The public repo has seven passing tests.
The lesson reaches past this tool. Anything that inspects untrusted repositories, whether a scanner, a CI check or an AI agent, has to treat the repo's own configuration as hostile. Running git in a folder you did not create is running code someone else wrote.
What Apiiro reported, as covered by The Hacker News
Apiiro examined 104,185 public copies of .claude/settings.local.json in 103,617 repositories and found secrets in 1,122 of them, holding 1,379 distinct secrets, across 63 owners. Counting older versions, the shared settings.json files and saved agent chat logs, the total reached 2,539 secrets in 1,994 repositories. Categories include 379 database passwords, 43 SSH logins, 42 Git and npm tokens and 23 AWS keys. Apiiro did not test whether the secrets still work. One company it notified confirmed its key was live. The Hacker News also reports that Claude Code adds the file to the user's global git excludes on first write but never to the project's own .gitignore, so the protection stays on one machine and does not travel with the project. We have not read Apiiro's own report. Those figures are theirs as reported.
Note the second file. The tool covers .claude/settings.json as well, since Apiiro's count of 298 secrets across 76,011 shared copies shows the shared file leaks too.
If it finds something
Rotate the credential first. Deleting the line or the file does not un-leak a secret that was already pushed or cloned. Then remove the entry, and if it was committed, purge it from history and force-push after rotation. Going forward, keep secrets out of commands and refer to them by variable name, and approve patterns instead of single commands. Add .claude/settings.local.json to your project's .gitignore.
A better fix, not yet built
The root cause is that the approved command contains the secret. The durable fix is to keep the secret out of the command altogether. The agent writes a reference such as bao://prod/admin-key. A hook resolves it from a secrets vault, such as OpenBao with the vault login protected by a hardware key that needs a physical touch, and swaps the real value in only for the one command that runs. The settings file then holds a reference, and there is nothing to leak.
This is a design, and we have not built it. We checked what Claude Code's documentation says, through a summary, so check the page yourself. A PreToolUse hook can return updatedInput, which replaces the command before it runs, and permission rules are checked against the rewritten input. A PostToolUse hook can replace a tool's output before the model sees it, which would let a hook scrub a secret that a command echoes back. The documentation does not say what "don't ask again" saves when a hook has rewritten the command, the original or the rewritten one. That is the whole question, because if the rewritten command is saved, the secret is back in the file. It does not say how to set a variable for a single command only, or whether hooks run in every permission mode. We will test those on a throwaway vault before we claim anything.
What the tool does not do
It matches known patterns, so an unusual token or a secret buried in a long script will be missed. It reads settings files on disk. It does not search shell history, session transcripts or backups, and it does not look in the other agent folders that The Hacker News suggests treating the same way, such as .cursor, .gemini and .specstory. It cannot tell whether a remote is public or whether anyone cloned the repo. Treat a finding as a lead.
Limits
We built and tested this ourselves. We have not run it against other people's repositories, and we do not know its false-positive rate beyond the placeholder case we hit. The Apiiro numbers are secondhand. The hook behavior comes from a summary of Anthropic's documentation.
Five percent of this is wrong somewhere. It always is. If the tool misses a secret shape, open an issue on the repository with the shape of the secret, and not the secret.
Every indicator in this post is in the feed. Free.
1.58M+ IOCs, STIX 2.1 / TAXII, 88% novel vs ThreatFox, exploited-CVE leads ahead of CISA. No credit card — a free API key in 30 seconds, and you can audit every claim above against the live endpoints.
Comments