Skip to content

Attribute anti-fraud and consent vendors, including by cookies and headers - #1206

Open
chrmod wants to merge 2 commits into
mainfrom
anti-fraud-cookies-headers
Open

Attribute anti-fraud and consent vendors, including by cookies and headers#1206
chrmod wants to merge 2 commits into
mainfrom
anti-fraud-cookies-headers

Conversation

@chrmod

@chrmod chrmod commented Aug 21, 2026

Copy link
Copy Markdown
Member

Adds the major bot-management vendors (Cloudflare, Akamai, DataDome, Imperva, PerimeterX/HUMAN, Kasada, AWS WAF) as anti-fraud, plus OneTrust/Cookiebot/IAB TCF consent cookies and AWS ELB.

Patterns can now also be matched by the cookies and response headers they set — attributing vendors a URL can't reveal (CNAME-cloaked / edge-integrated). New --- cookies / --- headers blocks (cookie names support a prefix* wildcard) and SDK matchCookie / matchHeader. See docs/response-signals.md.

chrmod added 2 commits August 21, 2026 16:10
…aders

Add anti-fraud coverage for the bot-management vendors that gate much of the web (Cloudflare, Akamai, DataDome, Imperva, PerimeterX/HUMAN, Kasada, AWS WAF), and let patterns be recognised by the cookies and response headers they set, so CNAME-cloaked and edge-integrated vendors a URL can't reveal are still attributed. Also covers OneTrust/Cookiebot/IAB TCF consent cookies and AWS ELB stickiness.
Pre-bucket the cookie and header indexes at export time (exact-cookie map, header buckets by name, wildcard-prefix map) so the matcher does a single hash probe per name; wildcard cookies resolve via a small prefix trie built at load, instead of scanning every entry.
@chrmod

chrmod commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Follow-up: reaching the browser extension (design note)

Heads-up for reviewers on where this does and doesn't land today.

The Ghostery extension does not consume this package's SDK — it loads the serialized TrackerDB engine and attributes requests via @ghostery/adblocker's engine.getPatternMetadata(request) / engine.metadata.fromDomain() (no @ghostery/trackerdb dependency). So the matchCookie / matchHeader added here (which read trackerdb.json) are reachable by SDK / CLI / WhoTracks.me consumers but not by the extension. The signals also don't reach the engine binary today: FiltersEngine.fromTrackerDB only ingests pattern.filters, and extra pattern keys are dropped on serialize — so cookies / headers never make it into engine.metadata.

To light this up in the extension, the matching has to live in the adblocker engine's metadata, folded into the existing attribution path (not the blocking match(), since cookies/response-headers are only known post-response). Recommended design:

  • Integrate into getPatternMetadata(request), not a public fromCookie/fromHeader. Add optional cookies / headers to the Request (fromRawDetails), and have getPatternMetadata fold cookie/header lookups into its existing unified IPatternLookupResult[] (same seenPatterns de-dup). One call returns URL + domain + cookie/header matches; the extension change is just attaching cookies/headers to the request it already builds.
  • Data in Metadata, keyed by fastHash(name) — the same string→pattern index categories/organizations already use (not the implicit NetworkFilter domain path). Exact names in a CompactMap; the handful of * wildcard prefixes in a small serialized list scanned at query time. fromCookie/fromHeader remain internal resolvers behind getPatternMetadata, not public API.
  • Carry cookies/headers on IPattern so merge() / fromTrackerDB keep working; extend fromTrackerDB to pass them through. This changes the metadata binary layout → ENGINE_VERSION bump.
  • Not modeled as $header / $cookie filters: the adblocker has neither today, and building filter syntax + a match bucket for what is fundamentally a name→pattern hash lookup is far more surface than a direct reverse index (URL tokenization buys nothing for exact/prefix names).

Change surface: mostly @ghostery/adblocker (IPattern, Metadata, Request, getPatternMetadata, fromTrackerDB, version bump); the trackerdb data/JSON already carry the fields; the extension is a couple of lines. The SDK matchCookie/matchHeader in this PR can be dropped once that lands, or kept for non-engine consumers.

Interim (if attribution is wanted before the engine change): ship the cookie/header index in trackerdb's dist and have the extension match it in getMetadata — same data, no engine-format change, retired once the adblocker path ships.

@chrmod
chrmod requested a review from philipp-classen August 21, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant