We Measured Our Own Free Edge Shield. The Blocklist and the Shield Catch Almost Entirely Different Attackers.

We run a threat intelligence platform, and we defend it with something that costs us almost nothing: a small script running on Cloudflare's edge, fed by a list of addresses our own feed rates as high confidence. No appliance. No agent. No console. No analyst watching a screen. We call it the edge shield, and the honest question we had never properly answered was whether it actually works, or whether it is security theatre that happens to be cheap.

So we measured it. This is what we found, including the part we got wrong.

The rule we made ourselves follow: count attackers, not requests

The easiest way to make any security product look spectacular is to count requests. A single scanner hitting your site four thousand times becomes "4,000 threats blocked!" on a dashboard. That number is real and it is meaningless. It is one attacker.

Every figure below counts distinct hosts instead. One machine is one machine no matter how many times it knocks. That single choice cuts our headline number by roughly seventy percent, and we would rather publish the smaller true number than the bigger flattering one.

The second rule: a block only counts as validated if somebody other than us also says that host is bad. We checked every blocked host against our own indicator corpus, which aggregates independent public and community sources. If nobody else has ever flagged it, we say so.

Finding one: the list and the shield barely overlap

Our blocklist held 2,167 addresses during the measurement window. The shield blocked 1,638 distinct hosts over the first sixteen days of August.

You would assume those are mostly the same machines. They are not.

Fifty-five hosts were in both. That is 3.4 percent of everything we blocked. Those fifty-five are the satisfying ones: our feed had already condemned them before they ever knocked on the door, so they were refused on arrival by reputation alone.

The other 96.6 percent were strangers. No feed we consume had ever heard of them. They were stopped on behaviour — what they did, how fast, from where, in what pattern — not on reputation.

This is the whole argument for putting a shield at the edge instead of just subscribing to a blocklist. A blocklist alone would have stopped 55 hosts. The shield stopped 1,638. That is roughly thirty times more, autonomously, with no human involved and at effectively zero marginal cost.

Finding two: 97.5 percent of our blocklist never fired, and that is the point

Here is a number that looks like failure until you think about it for ten seconds. Of the 2,167 addresses we had loaded and armed, 2,112 never showed up at all.

We had to resist the instinct to call that waste. It is not waste. It is coverage held in reserve. Those are known-bad machines that were pre-emptively refused entry, and they simply never chose to come. The correct reaction to "we were ready for an attack that never arrived" is relief, not a demand for a refund.

This reframes what a threat feed is actually for. A feed is not a prediction engine that should be judged on hit rate. It is insurance. The 3.4 percent that did show up is the claim you file; the 97.5 percent that did not is the coverage you were glad to have.

Finding three: 41.5 percent independently corroborated

Of the distinct hosts the shield blocked, 41.5 percent were independently confirmed as malicious by at least one source that is not us — public reputation feeds, community indicator exchanges, and our own honeypot infrastructure, which is genuinely first-party evidence.

We think that is a strong number for a system blocking primarily on behaviour, and we are deliberately not dressing it up as higher. The remaining ~58 percent is not proof of false positives; it is mostly rented cloud infrastructure that no feed has caught up with yet. Reputation lists are, structurally, always slightly behind. That lag is precisely the gap a behavioural shield exists to cover.

What we will not claim is a reputation-grade precision figure. We did not measure one, so we do not get to quote one.

What we found blocked, in aggregate

The shape of the traffic was consistent and unsurprising once we looked. The overwhelming majority of blocked hosts were rented cloud infrastructure — major public cloud platforms, budget hosting providers, and a handful of regional cloud operators — presenting themselves as ordinary desktop browsers while systematically walking our content.

That is the modern shape of the problem for a small site. It is not a hooded genius. It is somebody who rented forty machines by the hour and pointed them at you, and the giveaway is not the payload, it is the pattern: too regular, too fast, from a datacentre, wearing somebody else's clothes.

Sometimes those clothes are a consumer browser. Increasingly they are a crawler — we found a cluster of rented Google Cloud machines systematically identifying themselves as Amazon's web crawler, on the reasonable bet that most sites wave crawlers straight through without checking. Any defence built on trusting what a visitor calls itself is already defeated by this, and it is trivially cheap to do.

One cluster we blocked at the edge turned out to be the same infrastructure that was simultaneously trying to farm free API keys out of our registration form under fabricated identities. Same operator population, two different doors. Cheap defence at one door surfaced an attack on the other — which is an argument for logging your blocks somewhere you can actually query them later.

The part we got wrong — and then partly got wrong again

Measuring honestly means publishing the failures too, so here is ours. It comes with a twist we did not expect.

Our first pass found the shield blocking self-identifying web crawlers: Meta's crawler refused 332 times across 112 hosts while fetching our sitemap and article pages, plus Amazon's, Google's and Yandex's in smaller numbers. We wrote that up as four false positives and went to fix it.

Then we checked each one properly instead of trusting the user-agent string, and three of the four were not false positives at all.

Update, same day: we verified every blocked "crawler" against who actually owns the address it came from. The results below replaced our original claim of four wrongly-blocked crawler families. We are leaving the original framing visible rather than quietly improving our own numbers, because a correction that flatters you is exactly the kind that needs to be shown.

Claimed to be	Blocked	What it actually was
Meta's crawler	332 requests, 112 hosts	Genuinely Meta. Every single one from Meta's own network. A real false positive.
Amazon's crawler	245 requests, 28 hosts	Impostors. Every sampled host was a rented Google Cloud VM wearing Amazon's name.
Googlebot	8 requests, 6 hosts	Impostors. Bulletproof-hosting VPS boxes with no legitimate reverse DNS.
Yandex's crawler	7 requests, 5 hosts	Impostors. The same bulletproof hosts as the fake Googlebots.

So the shield was right about three of them and wrong about one. The scrapers had simply figured out that dressing up as a well-known crawler is a cheap way past a naive filter — and the shield caught them anyway, because it was not judging them on what they called themselves.

Our real error rate on crawlers was one operator, not four.

The genuine miss still matters. If your link previews stop rendering when customers share you, the free shield just became expensive. We had a known-good bot allowlist — it was enforced at our origin server, and blocking had since moved out to the edge. The allowlist never made the trip.

The fix is to verify crawlers rather than believe them. That means checking that the address really belongs to the operator it claims — and it means knowing that the right check differs per operator. Reverse DNS works for Google. It does not work for Meta, which publishes none, so a reverse-DNS test on Meta returns "unverified" for traffic that is completely genuine. That is precisely the mistake we made in our own second pass: we ran the wrong instrument and briefly concluded Meta's real crawler was fake too. Wrong instrument, not evidence.

Two lessons, and the second one is the expensive one:

A user-agent string is not identity. Most of what we block is already lying about being a browser. Treating "it says it's Googlebot" as proof is how you build a bypass instead of a shield.

We only found any of this because we went looking. Nothing alerted. The shield reported healthy the entire time, because from its own point of view it was doing exactly what it was told. A green dashboard is a claim, not evidence.

So is cheap security real security?

For this class of threat, yes — with an asterisk we are happy to say out loud.

A small business, a nonprofit, a local government office, or a solo developer running an application edge can put a behavioural shield in front of their site for approximately the cost of nothing, and it will autonomously stop thousands of hostile automated hosts a month that no blocklist would have caught. It requires no security team, no tuning, and no one watching. That last property is what actually makes it viable for organisations that will never hire an analyst.

The asterisk: it will not stop a targeted human adversary, it does not replace patching, and if you deploy it without checking what it blocks, it will eventually eat something you needed. Ours did.

We think that trade is overwhelmingly worth it for the people who currently have nothing, which is most people. The security industry sells extremely good products to organisations that can afford a security team. The organisations that cannot afford one are not less attacked — they are just less defended, and less measured.

Our numbers, stated plainly

Distinct hosts blocked in sixteen days: 1,638. Addresses armed on the blocklist: 2,167. Hosts appearing in both: 55, or 3.4 percent. Blocklist entries that never had to fire: 97.5 percent. Independently corroborated as malicious: 41.5 percent. Crawler families wrongly blocked: one, out of four we initially suspected — the other three were impostors the shield was right to stop. Human interventions required during the measurement window: zero. Marginal cost: effectively zero.

We cap our confidence at 95 percent on principle, because something is always wrong somewhere and claiming certainty is either a lie or ignorance. But we will stand behind the direction of every number above, and we will publish the corrections when they come — as we just did.

If you run something small and you have nothing in front of it, put something in front of it. Then measure it, with a denominator, and be prepared to find out that it is blocking Google.

---

How do AI models see YOUR brand?

AIPM has audited 250+ domains. 15 seconds. Free while still in beta.

Audit your site now → aipmsec.com