How-to

The evidence-gated merge check

A GitHub check that maps a pull request's changed files to the test cases behind them, and asks whether they passed with evidence on this commit.

Applies toPlans:FreeProBusinessEnterpriseRoles:OwnerAdminDeveloperWhere:Web appMCP

"Definition of done" is usually a sentence in a wiki. This turns it into something a repository can check: not "tests exist", not "CI was green", but "the specific cases linked to what you touched were proven on the commit you are asking to merge".

Turn it on

Connect GitHub for the project if you have not already — see connect GitHub.
Push a codebase map so files resolve to features: bugmojo map --features from the repository root.
Make sure your runs are being recorded — the gate needs results on a commit, which is what the reporter produces.
Open a pull request. The BugMojo evidence check appears with a summary and a comment listing what is missing.
Leave it advisory for a couple of weeks. Switch it to required in the project's settings once the coverage it reports matches what you expect.

How the verdict is reached

Four narrowing steps

Changed files. The pull request's file list, with generated and irrelevant paths filtered out.
Files → features. Each remaining path is matched against the feature ledger's source references — routes, API procedures, components, flows.
Features → cases. The accepted, non-deleted test cases linked to those features.
Cases → results. The newest result per case, computed twice: once restricted to this commit, and once overall — so the check can distinguish "proven here" from "last proven on an older commit".

That last distinction is the whole point of the gate, which is why both are computed even though only one of them can make the check pass.

Reading the result

What the check is telling you

Green — every linked case passed on this commit
Likely cause: Each case behind the changed files has a passing result on this SHA with at least one evidence row.
Nothing to do. The summary lists which cases counted, so a reviewer can see what "green" covered.
Red — cases have no result on this commit
Likely cause: The suite has not run against this SHA, or the reporter is not wired into the job that ran.
Run the cases and record them. The pull-request comment prints the exact bugmojo run command for the cases that are missing.
Red — a linked case failed
Likely cause: A case behind your change is failing on this commit.
Open the result, read the trace and video, and either fix the change or triage the failure as FLAKY, ENVIRONMENT or TEST_DEFECT with a reason.
Red — a case passed but has no evidence
Likely cause: A result was recorded without artifacts, usually because trace, video and screenshot were all disabled in the Playwright config.
Set trace: 'retain-on-failure' and screenshot: 'only-on-failure' so a result carries something a human can check.
Skipped with a reason
Likely cause: No changed file resolved to a feature, or no accepted case is linked to the features that matched.
Re-run bugmojo map --features, or link cases to the feature. A skip with a reason is deliberate — a silent green would be a lie.

Scope and limits

Evaluation is bounded so a huge repository cannot make a pull request wait: there are explicit ceilings on the number of features scanned, the number of cases one pull request is gated against, and the number of results read per commit. When a ceiling is reached, the check says so rather than pretending it saw everything.

The configuration lives per project, so one repository can be advisory while another is required. A malformed or missing configuration resolves to the documented defaults — a config parse error must never block a merge.

Common questions

Will this turn my repository red on day one?

No. The check is advisory by default — it reports and cannot block. Switch it to required per project once you trust the coverage it shows.

What if a changed file maps to no feature?

The check skips with a reason rather than passing silently. A skip that says why is honest; a green tick that means "we found nothing" is not.

Does a pass from last week count?

No. The gate looks for the newest result on <em>this</em> commit with at least one evidence row. An older pass is reported separately as context.

How do I map files to features?

Run <code>bugmojo map --features</code> to push a codebase map — routes, APIs, components and flows with their source paths — into the feature ledger.

Can a manual case satisfy the gate?

Yes. A manual result carries the same evidence rows as a CI result, so a human execution on that commit counts exactly as an automated one does.

Was this helpful?

Updated Sep 19, 2026 · Still stuck? Contact support