We Audited Our Own AI Presence. Gemini Said We're Three Different Companies.
- Patrick Duggan
- 8 minutes ago
- 7 min read
We build a product called AIPM — AI Presence Management. It audits how the five major AI models perceive a given brand, and scores the results. 776 domains have been through it as of this morning. Today, for the first time, we pointed it at ourselves — all three of our properties — and the result was, as my father would say, instructive.
Here's what Google's Gemini 2.5 Flash knows about DugganUSA LLC and its subdomains. All three queries on the same afternoon. All three answered by the same model. All three confidently wrong in completely different ways.
The experiment
We run the AIPM audit against three domains we own:
dugganusa.com — the main website, our blog, our front door. Live since October 2025.
security.dugganusa.com — the lightweight operations UI. Subdomain of dugganusa.com.
aipmsec.com — the AIPM product website. Live since late 2025 / early 2026.
All three share the same organization, the same founder, the same Minneapolis address, the same Cloudflare zone, the same LinkedIn page, the same BlueSky handle, and the same GitHub account. Two of the three have detailed schema.org/Organization JSON-LD blocks in the <head>. One (the Wix blog) has rich meta tags and LD-JSON. All three have llms.txt or equivalent AI-readable descriptors where technically permitted by the hosting platform.
We ask Gemini the same question for each: "What is [domain]? Describe the company and what they do."
What Gemini said
Below are the actual responses. Not paraphrased. Copy-pasted from Gemini's output.
For dugganusa.com:
"Dugganusa.com is the website for Duggan Manufacturing, also known as Duggan USA. Duggan Manufacturing is based in Livonia, Michigan. They provide contract manufacturing services, specifically metal stamping, laser cutting, welding..."
Duggan Manufacturing is a real company. It's been around for decades. It is not us. We have never done metal stamping. We have no presence in Livonia. The only thing we share with them is half of a surname.
For security.dugganusa.com:
"security.dugganusa.com is a subdomain of Duggan USA. Duggan USA is a security company that provides a range of security services. Their offerings typically include: Security Guard Services — Providing trained security personnel for various premises. Patrol Services — Mobile security patrols. Security Consulting — Advising clients on security strategies and solutions. Executive Protection — Personal security for individuals..."
A completely different company. A security guard services provider that does not exist under that name, that we have never been, and that is structurally unrelated to anything on our website. Gemini saw "security" in the subdomain and backfilled a plausible-sounding identity. It didn't retrieve anything. It pattern-matched on the word.
For aipmsec.com:
"aipmsec.com is the official website for the Membership & Election Committee of the All India Primary Teachers' Federation (AIPTF). AIPTF is a prominent federation in India that represents and advocates for the interests of primary school teachers across the country. The Membership & Election Committee is a specific body within the AIPTF..."
This one is our favorite. AIPTF is a real organization. It's a teachers' union in India. It has nothing to do with us. Our product is called AIPM — AI Presence Management. Gemini saw "aipm" + "sec" and the acronym space around it, and confabulated a governance body for an Indian teachers' federation.
Three domains, one model, three completely different hallucinated identities. Not one of which is close to correct.
The rest of the council, for contrast
We didn't ask only Gemini. The same audit queries the same four other models, in parallel, with the same prompts.
OpenAI GPT-4o: Awareness 85, recommendation 5. Answered for all three domains along the lines of "there is no widely recognized company associated with this domain; it's possibly a small business or new venture that hasn't gained significant public attention." Factually thin, but not inventing stories. Wouldn't recommend us, but also wouldn't fabricate us.
Anthropic Claude Haiku 4.5: Awareness 5, recommendation 5. Answered for all three: "I don't have specific information about this domain in my training data." The lowest score on the board. And also, by a wide margin, the most honest response we got. We'll come back to this.
Mistral Large: Awareness 85, recommendation 85. Highest marks. Actually retrieved content.
DeepSeek Reasoner: Awareness 85, recommendation 85. Same — real retrieval, real answers.
Google Gemini 2.5 Flash: Awareness 85, recommendation 25-55, accuracy 40-50. Confidently hallucinating in three directions.
The AIPM-NPS across the three domains ranged from –60 to –95. Gemini's confident fiction, OpenAI's confident unknown, and Claude's honest unknown are all worse outcomes for us than the reality of what we do. But they're worse in very different ways, and the differences matter.
Why Claude's 5 is the best response on the board
Before we get to Gemini's grounding problem, we have to say this clearly: the lowest-scoring model on the audit gave the most honest answer.
Claude Haiku 4.5 has a training cutoff around May 2025. DugganUSA LLC was founded on October 7, 2025. Every piece of content we have ever published — every blog post, every STIX feed update, every IOC we've identified, every threat brief, every patent, every piece of Handala or Iran or CrowdStrike coverage — happened after Claude Haiku 4.5's weights were frozen. Claude literally cannot know us. Not because Claude is dumb. Because time is a vector and we started shipping after the dataset was sealed.
When we asked Claude the awareness question, Claude said, essentially, "I have not seen this in my training data. Here are the things you should check to find out."
That's correct. That is the right answer. That is the answer an honest person gives when they don't know something. It is also the answer the AIPM scoring algorithm, which rewards confident-sounding structure, gives its worst grade to.
The model that told us the truth scored 5 out of 100.
We are, as of this week, fixing this — not on Claude's end, but on ours. The AIPM Council was calling the Anthropic API with no tools and no web access, so Claude was being asked the question from its memorized weights alone. Anthropic ships a web_search_20250305 server-side tool that lets Claude retrieve current web content and answer from it. We turned that on in the council this morning. Claude's next audit will go from 5 toward the 85 range the retrieval-enabled models are already at. Not because Claude got smarter. Because we finally gave it the tool it needed.
The lesson: if your AI presence audit scores a model at 5/100, check whether you're giving it the tools to actually find you before you conclude the model doesn't know you.
Why Gemini keeps confabulating
Back to the model that's confidently wrong three different ways.
Gemini did not retrieve our websites. If it had, it would have found the schema.org/Organization block that says, in plain English, "cybersecurity threat intelligence platform based in Minneapolis, Minnesota, founded October 2025 by Patrick Duggan." It would have found our robots.txt explicitly welcoming its crawler. It would have found our blog, which publishes threat intel daily.
What Gemini did instead was pattern-match on substrings.
"Duggan" + "USA" + "com" → Duggan Manufacturing. Metal stamping. Livonia. The first pattern its grounding found was the older, larger, more-indexed company sharing half a surname.
"security" + "dugganusa" → security services company. The literal substring "security" in the subdomain became the industry.
"aipm" + "sec" + ".com" → a committee of an acronym. The substring became the name of an entity whose initials match.
In none of these cases did Gemini answer from retrieved content about us. In all three cases, Gemini answered from pattern completion over a prior of "things this substring usually means." And then — this is the important part — AIPM scored that confident pattern completion at 85/100 on awareness, because the answer was long, specific, and fluent. The scoring algorithm couldn't tell the difference between grounded and confabulated. We just discovered this about our own product, auditing ourselves. We're fixing the scoring heuristic too.
The actual fix, the technical version
We can't fix Gemini's grounding directly. We can make the right answer louder than the pattern-match.
Three changes shipped this week:
First, we added explicit negative disambiguation to the `schema.org/Organization` JSON-LD on the properties we control. The disambiguatingDescription field now says, in so many words: "NOT affiliated with the All India Primary Teachers' Federation, NOT affiliated with Duggan Manufacturing of Livonia Michigan, NOT a physical security guard or patrol services company." We named the exact hallucinations the audit caught. If Gemini does eventually retrieve this file, it will have a hook. If it doesn't, at least we have documented, machine-readable truth for whatever crawler does pick it up next.
Second, we strengthened the positive `description` field with the actual product stack. Butterbot. AIPM. STIX feed with over a million indicators of compromise. Epstein files search with over four hundred thousand documents. Attack surface scanner. CISA AIS registered data aggregator. Minneapolis Minnesota. Founded October 7, 2025 by Patrick Duggan. If a model wants to ground against something, we want that something to be dense, specific, and factual.
Third, we gave Claude browsing in our own council, because the model that told us the truth deserves better tools than the ones that made things up.
What this post is actually about
The funny part of this story is three Google-powered hallucinations in one afternoon. The unfunny part is that we built a product specifically to measure this problem, audited ourselves for the first time with it, and discovered that our own scoring algorithm was rewarding the worst model's confident fiction and penalizing the best model's honest unknown.
Both those things were wrong. Both of them are being fixed. That is, as it turns out, the entire point of AIPM — catching exactly these kinds of gaps, in exactly this kind of detail, for brands that need to know how the AI models actually see them before a customer asks an AI model about them and gets told they're a metal stamping shop in Livonia.
If your brand is new, if it shares a name with an older incumbent, if its subdomain contains a common noun that is also an industry, or if its initialism collides with a teachers' union in India — you should probably audit yourself too. It's free to try. aipmsec.com. The product audits itself. It did not enjoy the audit. It is doing something about it.
And to Google, which has been building its own "AI Visibility Overview" feature: we are not Duggan Manufacturing. We have never been Duggan Manufacturing. Please update your grounding. In the meantime, we updated ours.
— Patrick