```html ```
top of page

A Reader Asked for MISP. We Shipped It in a Day — and the First Version Would Have Republished Spamhaus's Work Under Our Name.

  • Writer: Patrick Duggan
    Patrick Duggan
  • 1 hour ago
  • 5 min read

Updated: 19 minutes ago

We now publish a MISP feed. Point a MISP instance at https://analytics.dugganusa.com/api/v1/stix-feed/misp/ — Sync Actions, Feeds, Add, format "MISP Feed" — with an API key. It is live as of today, and free.


The more useful half of this post is what went wrong building it, because the first working version was quietly doing something we spend a lot of words telling other people not to do.



Why MISP, and how we found out


We ship STIX 2.1, TAXII 2.1, CSV blocklists and native OPNsense feeds. We did not ship MISP, which is the default sharing platform for CERTs, ISACs, national teams and a large slice of non-US public sector — frequently organisations that cannot consume TAXII but run a MISP box.


Three independent signals, none of them a roadmap meeting:


A reader in Australia left a 10/10 response in our feedback widget with the comment "MISP is about all, but certainly can work with the other formats." He had found us through Clint Gibler's newsletter, read enough to have an opinion about ingest formats, and told us.


Someone else had said the same thing earlier.


And then, reviewing seven days of edge logs for something else entirely, we found a live MISP 2.5.43 instance polling /api/v1/stix-feed/manifest.json — the exact path a MISP feed manifest lives at — and receiving a 401 for a route that did not exist. Somebody had already configured a client and pointed it at us, on the assumption the feed would be there.


That is three signals for one format, and the third one had been knocking for a while.



The build is genuinely small


A MISP feed is not the MISP application. It is a static tree over HTTPS: a manifest.json mapping event UUID to a summary, plus one <uuid>.json per event. It is a serialisation of data we already publish, on the same route and the same API-key middleware as the CSVs.


One event per campaign source — which is exactly why our own campaign rule forces a campaign-specific source at ingest time. Event UUIDs are derived deterministically from the source name, so when MISP re-polls the manifest it recognises events it already has instead of treating every poll as a fresh disclosure.


Then we read the output, and it was wrong four times.





Defect one: we were laundering


The first live manifest returned five events. All five were third-party feeds — Spamhaus DROP, ThreatFox, OpenPhish, SSLBL, MalwareBazaar — published as MISP events carrying Orgc: DugganUSA.


We have a hard rule against exactly this. Never launder third-party feeds as a DugganUSA observation; cite corroboration, never re-ingest under our own source. We have written it about Spamhaus specifically. And the first version of a new distribution channel did it automatically, at scale, into other people's MISP instances — where it would arrive looking like our collection.


It happened for a boring reason. The grouping sorted by recency, and third-party syncs run continuously at high volume, so the newest records are essentially always somebody else's feed. Nobody decided to republish Spamhaus. The default did.


It is also bad product, which is worth separating from the ethics. A MISP operator subscribing to us almost certainly already pulls abuse.ch and Spamhaus directly. A mirror of their feed under our name adds nothing to their coverage and costs us the credibility of everything sitting next to it in the same event list.


The MISP feed now carries first-party and curated sources only: our own hunting, campaign ingests, adversary back-fills, research imports. Third-party corroboration stays in the CSVs and the STIX bundle, where the per-record source field travels with the data and attribution is never lost.



Defect two: then it returned nothing


Filtering the third-party sources out of a recency scan produced an empty manifest. Zero events.


Same cause, other direction. If you scan the most recent N records and then filter, and one class of writer is a thousand times noisier than the other, the quiet class is never inside the window. A campaign we had ingested ninety minutes earlier was already buried.


Recency is the wrong axis when the writers are that unequal. The fix was to stop scanning and start asking: enumerate the distinct sources, drop the mirrors, then query each surviving source directly. That also surfaced a ceiling worth stating — the enumeration returns at most 100 distinct sources, so the feed logs how many it saw versus how many it published rather than silently truncating.



Defect three: we were about to weaponise our own doormat


With real data flowing, the largest event was edge-honeypot — two thousand indicators — and every one was marked to_ids: true, meaning "safe to build a detection rule from."


Those are honeypot visitors.


A honeypot exists to get hit. Everything that touches it is a visitor, and a visitor is not a block candidate — the value in that data is the free per-hit metadata, not the source address. Publishing two thousand of them with the IDS flag set means every downstream MISP turns our doormat into their firewall, blocking researchers, scanners, shared NAT egress and anyone who happened to knock on a machine that exists to be knocked on.


That is our own rule, applied to somebody else's network, at a scale where nobody would have traced the resulting false positives back to us. Excluded. Test-fixture sources went with it, for the obvious reason.



Defect four: the same IP, five times


Our indicator store is a point-observation model — one document per sighting, deliberately, so we keep the timeline. Mapped straight into MISP, that meant a single address arriving as five identical attributes.


MISP would have stored all of them, and every downstream correlation would then have been weighted by how often we happened to be looking rather than by what the indicator is. Deduped by value, newest sighting wins. Verified on the live endpoint: two thousand raw records became 1,384 unique attributes with zero duplicates.



What it looks like now


Forty events. Zero third-party mirrors. No honeypot data. No test fixtures. No duplicate attributes. Indicators mapped to real MISP types — ip-dst, domain, hostname, url, md5, sha1, sha256 — with to_ids set from confidence.


Two of our types have no clean MISP equivalent: malicious packages and on-chain dead-drop addresses. Both degrade to text with to_ids: false rather than being forced into a type that looks close enough. A mislabelled attribute becomes a bad IDS rule on somebody else's network, and a comment is better than a wrong answer.



The part worth taking away


Every one of those four defects was invisible in the code and obvious in the output. The code was correct — it did precisely what it said. It took reading the actual JSON a customer would receive to notice that correct code was producing a doctrine violation, then an empty file, then a weaponised doormat, then inflated correlation weights.


We write constantly that green is a claim rather than evidence, and that you have to check the artifact rather than the status code. This is that, applied to ourselves, four times in one afternoon on a feature that took a day to build.


If you run MISP, the feed is there and it is free. If you find something in it that should not be — a mirror we missed, a source that has no business being IDS-flagged — tell us and we will publish the correction, which is the other half of the job.


Ninety-five percent, as always. One known cosmetic gap: the manifest's human-readable counts are computed before deduplication, so they read higher than the event delivers. The event is authoritative; the count string is being fixed.




How do AI models see YOUR brand?

AIPM has audited 250+ domains. 15 seconds. Free while still in beta.


Comments

Rated 0 out of 5 stars.
No ratings yet

Add a rating
bottom of page