We Nearly Re-Indexed Two Million Records to Fix the Wrong Thing
Our search engine thought 9.9.9.9 and 47.86.9.9 were related. They are not. One is a public DNS resolver run by a Swiss non-profit; the other is a command-and-control server.
The reason is mundane. Search engines tokenize on punctuation, and a dot is punctuation. So the address 9.9.9.9 is not stored as one thing. It is stored as four nines, and a query for it matches any record containing a nine in any position.
We had worked around this three separate times in application code without ever asking whether the index could simply be told not to do it.
There is a settings field for this
There is. It controls which characters are treated as word separators, and adding the dot to the exceptions means an address is stored as a single token.
We tested it on a scratch index holding eight documents. It was perfect. Queries for 9.9.9.9, for 117.55.202.115 and for 185.39.19.176 each returned exactly themselves and nothing else. Three documented incidents from our own history, apparently resolved by one settings field we had never set.
That is the point at which the temptation is to ship it.
Why we could only do this once
The index in question holds 2,025,493 records. Changing a tokenizer setting re-indexes the entire corpus, and while that runs it holds an exclusive write lock. We know exactly what that costs because we have paid it: a settings change in May enqueued hundreds of re-index tasks and starved roughly eight thousand pending writes behind them. The queue monitor for that system has been in breach recently.
So this is a one-shot change on a working production index, with no cheap way to iterate and no quick way back.
One shot is exactly the condition under which you stop trusting a test that agreed with you. Eight documents will confirm whatever you were hoping. We built a gate instead: clone three thousand real records from the live corpus into a throwaway index, define the questions a correct engine must answer, and measure them before and after.
The questions were: does an address query find that address, how many other addresses come back with it, does a query for a bare address wrongly return a network block, can a parent domain still find its subdomains, and do hashes stay unaffected.
Four configurations, three thousand records
We made three predictions going in. All three were wrong.
We expected the dot setting to eliminate the false matches. It reduced them from 2,634 to 610. That is a large improvement and it is not the clean result the scratch test promised. Had we shipped on the strength of eight documents, we would have reported a 77% reduction as a success without ever learning what the remaining 610 were.
We expected it to fix address-versus-block confusion. A query for a bare address was returning the network block that contains it — the exact false positive that forced us to write a dedicated matching library a few weeks ago. The setting changed that number by precisely zero. The reason is embarrassing once you see it: the slash is also a separator, so a block like 185.39.19.0/24 still splits into the base address and the prefix length, and querying that base address still matches it. We had reasoned about dots and forgotten slashes.
We expected turning off typo tolerance to be the cheap, safe half. Typo tolerance is the feature that rescues someone searching for "restarant". Applied to identifiers it is actively harmful: a query for 185.39.19.176 was returning 185.39.19.177 and 185.39.19.17, because as single tokens those are within an edit or two. Disabling it on the indicator field does not touch tokenization, so it should have bought accuracy at no cost to anything else.
On its own it did nothing. Identical numbers, to the digit.
That one took a minute to understand and it is the most interesting result of the day. When dots are already splitting the value, the octets match each other exactly. There is no typo to tolerate. Typo tolerance is never consulted, so disabling it changes nothing. It only becomes relevant once the whole address is a single token — which is why the two settings together get the false matches down to four, while either one alone leaves you at 610 or 2,634.
Two mechanisms, stacked, where we had diagnosed one. The small test could not have shown us that, because with eight documents there are no near neighbors to confuse.
The cost we were not willing to pay
Look at the last column.
Keeping the dot takes domain-family search from 40 out of 40 to zero out of 40. Not degraded. Gone.
The reason is simple and we did anticipate this one: with dots preserved, a subdomain is a single token, and the parent domain is a suffix of it rather than a prefix. Searching for a site no longer finds its own subdomains. For a corpus where one attacker-controlled domain can carry sixty-one related names, that is not a rounding error.
So there is no configuration that wins both columns. Clean address lookups and working domain families are mutually exclusive at the index level. You pick.
Which is why we are not changing anything
We already have both, and we get them by not asking text to do this job at all.
For "do we hold this address", an exact lookup returns zero false matches — not four, zero — because it compares values instead of matching tokens. For "what else is on this site", a filter on the registrable domain name is exact set membership rather than substring luck, and it returned sixty-one genuine siblings for an attacker domain earlier the same week.
The application code we wrote to work around a bad default turns out to be better than the best setting we could have used instead. That was not the conclusion we expected, and it is worth more than the change would have been: we now know, with numbers, that the workaround is not technical debt waiting to be paid down. It is the right answer.
The part that generalizes
A search engine's defaults are tuned for prose. Our corpus is identifiers — addresses, network blocks, file hashes, vulnerability IDs, package names. Nearly every default that helps natural language hurts us:
Typo tolerance rescues a misspelled word and corrupts an address. Splitting on dots is right for the end of a sentence and wrong for anything dotted. Prefix matching makes autocomplete feel instant and makes exact lookup unreliable.
None of this is a flaw in the engine. It is a mismatch between what the defaults assume and what we store, and no vendor documentation is going to warn you, because the documentation is not about your corpus. The vendor does document disabling typo tolerance for product codes and serial numbers — the advice exists. What does not exist anywhere we could find is how dotted identifiers tokenize, or that these two settings interact in a way that makes either one alone misleading.
If you run a search index over identifiers rather than sentences, the defaults are probably costing you accuracy right now, and the only way to find out is to measure your own data.
We are publishing the harness for that reason. It samples real records from a live index into a throwaway probe, runs a held-out gate, applies the setting, re-runs, and deletes the probe. It refuses to start if the task queue is busy. The whole thing takes about six minutes and it saved us from an irreversible change to two million records that would have made two of three things worse.
The accounting
This post makes 17 checkable factual claims, all measured directly: every number in the table, the corpus size, the GitHub-documented behavior of the settings, and the engine version.
The honest headline is that we got three predictions wrong in a row and the cheap test caught all three. The expensive version of this story is the one where the eight-document test was the only test, the change shipped, and we spent a week wondering why domain searches had gone quiet.
We cap ourselves at 95% on anything. Something here is wrong, because something always is. When we find it, we will say so at this address.
Measure twice. The second measurement is the one that disagrees with you.
Every indicator in this post is in the feed. Free.
1.58M+ IOCs, STIX 2.1 / TAXII, 88% novel vs ThreatFox, exploited-CVE leads ahead of CISA. No credit card — a free API key in 30 seconds, and you can audit every claim above against the live endpoints.
Was this useful? Thirty seconds, no cookies, no tracking, no third parties, your address hashed and never stored. If the box below does not load, the same question lives at https://analytics.dugganusa.com/nps.html?post=search-defaults-assume-prose-our-corpus-is-identifiers-october-2026
Comments