Deduplicating Bug Reports: Fingerprinting Errors Before They Pile Up

An unfiltered error stream is mostly the same handful of bugs, repeated. One null dereference on a hot checkout route can fire ten thousand times in an hour, and if each occurrence lands as its own row, your triage queue is now ten thousand rows deep for what is, in truth, one defect. Deduplication is the mechanism that collapses those occurrences into a single issue. The hard part is not the counting — it is deciding, deterministically and at write time, whether a brand-new error belongs to an existing issue or starts a new one. That decision is a fingerprint: a compact key computed from each event so identical defects hash to the same value and genuinely different ones do not. This post is the build guide: what to put in the fingerprint, how to normalize the inputs so they survive a deploy, exact versus fuzzy matching, the merge-versus-split heuristic, and how to surface the result so triage actually scales.

This is not a novel idea — it is the settled core of every production error monitor, and the prior art is worth copying. BugSnag states its default plainly: it groups by the error class, file, and line number of the top in-project stack frame of the innermost exception. That is a hand-built fingerprint from type plus normalized top-of-stack location, which is exactly the recipe a homegrown deduper should reach for. Sentry's grouping is more layered but has an explicit precedence: it considers the fingerprint first, the stack trace next, then the exception, and finally the message. The lesson from both: a stack trace is the strongest signal you have, and the message string is the weakest — messages embed timestamps, IDs, and interpolated values that vary on every single event.

A naive fingerprint — hash the raw stack trace string — forks into a new issue on every deploy, because line numbers shift, minified frames are gibberish, and the route carries a fresh UUID each time. Normalization is the work that makes the key stable. Do it in this order before you hash anything:

• Resolve source maps. A minified frame like a.b@app.4f3c.js:1:88421 is meaningless and changes every build. Map it back to Checkout.tsx › submitOrder first.
• Strip line and column numbers from the frame used in the key. They move whenever anyone edits the file above the throw site. Keep the file and function; drop the position.
• Keep only in-app frames. Sentry groups solely on frames the SDK marks as your application, so two events that differ only in library frames still merge. A React or lodash upgrade should not fork your issue.
• Canonicalize the route. Turn /orders/9a3f-uuid?ref=email into /orders/:id. Strip query strings, replace UUIDs, hashes, and numeric IDs with placeholders.

What survives normalization is the load-bearing part of the event. What you stripped was noise that would have splintered one defect into thousands of singletons.

The null-byte separator matters: joining on a plain character lets ("ab", "c") and ("a", "bc") collide into the same key. Hash a tuple, not a concatenation. With this function, a write-time lookup is one indexed query — SELECT id FROM issues WHERE fingerprint = $1 — and you either increment the existing issue's counter or insert a new one. That is the entire exact-match deduper.

Exact-match fingerprinting is fast, deterministic, and trivially debuggable: two engineers can compute the key by hand and agree. Its weakness is brittleness — if normalization misses one volatile detail, near-identical events still split. Fuzzy grouping addresses that by comparing events on similarity instead of equality: token overlap or edit distance over the frames, or embeddings of the message and stack. Anything above a threshold merges. The cost is non-determinism and a threshold you must tune: too loose and unrelated errors collapse together, too tight and duplicates leak through. The pragmatic production pattern is to do exact-match first for the bulk of traffic, then run a fuzzy second pass over the stragglers — the events that landed as singletons — to catch the near-duplicates a strict hash missed.

The hardest calls sit on the boundary. The heuristic that travels well: would one code change close both occurrences? If yes, they should share a fingerprint. A TypeError thrown from the same submitOrder function on the same /orders/:id route, where only the message text or the library frames below differ, is one bug — merge it. The same TypeError fired from two different call sites or two different routes is usually two bugs needing two fixes — split it. The control you reach for is not a global similarity slider; it is the depth of in-app frames you fold into the key. Folding only the top in-app frame merges aggressively; folding the top three splits more finely. Over-merging buries a fresh regression inside a giant catch-all bucket where nobody notices the count ticked up. Over-splitting rebuilds the exact noise you were trying to deduplicate. Tune the frame depth per project and leave the slider alone.

Duplicate volume is not a universal constant — it is wildly project-dependent, which is the empirical reason a one-size threshold misbehaves. Across the 150,000-plus reports in the GitBugs dataset, duplicates run as high as 28.2% in VS Code and 20.9% in Mozilla Core, but as low as 2.0% in HBase. A grouping aggressiveness tuned for VS Code's repeat-heavy stream will over-merge HBase; one tuned for HBase will under-merge VS Code and leak duplicates back into the queue. Make the frame depth and any fuzzy threshold per-project knobs, not platform constants.

Deduplication only pays off if the collapsed count is visible. The grouped issue should carry a seen N times counter, a first-seen and last-seen timestamp, and an affected-user count. Those four numbers convert an unbounded chronological list into a ranked queue: the team fixes the issue hitting 4,000 sessions before the one hitting three, and a first-seen that lines up with last night's deploy instantly flags a regression. Without the counter, a triager re-reads the same defect dozens of times before reaching a genuinely new one — which, at GitBugs' 20-28% duplicate rates, is most of the queue.

Grouping logic is not static, and changes to it are dangerous if applied backward. Sentry's policy is the one to copy: each time default error grouping behavior is modified, it is released as a new version, applied only to new events going forward. Re-fingerprinting old events would silently re-shuffle every existing issue — splitting buckets people have been watching, merging ones they triaged apart, and resetting every counter. Freeze the algorithm version per project, stamp each event with the version that grouped it, and apply new logic only to incoming reports. If you must correct grouping for an existing issue, Sentry and BugSnag both offer manual merge plus a custom fingerprint rule that overrides the default — a targeted override, not a global recompute.

Here is where every prior-art system stops, and where BugMojo goes further. Sentry, BugSnag, and Rollbar all give you grouping plus a config UI for tuning stack-trace and fingerprint rules — but that UI is human-only. The grouped issue is a static artifact you read and a settings page you edit. BugMojo exposes the deduplicated issue — its fingerprint, its seen N times count, and the attached rrweb replay, console, and network bundle — over an MCP server. That means an AI agent (Claude Code, Cursor) can read the grouped issue, judge whether a borderline event was mis-merged or wrongly split by comparing the actual replays behind it, and propose a fingerprint or stack-trace rule change as code in a pull request. No competitor ships an agent-readable layer over the grouped bundle. Grouping today is an algorithm with a settings page; BugMojo makes it something an agent can inspect and correct. To be honest about the tradeoff: BugMojo is pre-release and in-session capture, not a fleet-scale production exception monitor — for aggregating uncaught exceptions across millions of prod sessions, a dedicated error monitor still wins that row.

• Resolve source maps before reading any frame for the key.
• Drop line and column numbers from the frame; keep file and function.
• Filter to in-app frames only so library upgrades do not fork issues.
• Canonicalize the route — strip query strings, replace UUIDs and numeric IDs with placeholders.
• Hash a tuple (type + top app frame + route) with a null-byte separator, not a string concat.
• Lookup at write time on an indexed fingerprint column; increment or insert.
• Run fuzzy as a second pass over singletons, with a per-project threshold.
• Surface the four numbers: seen count, first-seen, last-seen, affected users.
• Stamp a grouping version on every event; never recompute history.
• Expose the grouped bundle over MCP so an agent can audit merges and propose rule changes.