How-to

Record Playwright runs with the reporter

Three lines in playwright.config.ts and two environment variables turn every test run you already do into recorded, provable results.

Applies toPlans:FreeProBusinessEnterpriseRoles:OwnerAdminDeveloperQaWhere:Web appMCP

Install

Terminal

npm install -D @bugmojo/playwright-reporter
# or: pnpm add -D @bugmojo/playwright-reporter

Configure

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['@bugmojo/playwright-reporter']],
  use: {
    trace: 'retain-on-failure',
    video: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
});

Terminal

BUGMOJO_API_KEY=bm_key_...   # Settings → API keys, with write permission
BUGMOJO_PROJECT=WEB          # your project key

Point a spec at its test case

Four conventions are supported; use whichever suits your repo. The annotation survives renaming the file, which is why it is the one we recommend.

test('checkout with a saved card', async ({ page }) => {
  test.info().annotations.push({ type: 'bugmojo', description: 'WEB-42' });
  // ...
});

Convention	Example
Annotation	`{ type: 'bugmojo', description: 'WEB-42' }`
Filename	`tests/bugmojo/WEB-42.spec.ts` · `WEB-42-checkout.spec.ts`
Title prefix	`test('[WEB-42] checkout with a saved card', …)`
Tag	`test('checkout @WEB-42', …)`

What lands in BugMojo

After one run	Where it lands
A run stamped with the commit SHA, deployed URL, environment and runner (LOCAL or CI)	the project's run history
One result per test — status, duration, retries, the first line of the failure	pinned to the case's exact revision
trace.zip, video, screenshots, stdout and stderr, and anything you attached	evidence on the result, replayable in the viewer
A PROPOSED case plus a bug for a failure with no matching case	the review queue, written up already
Flaky tests — failed, then passed on retry	the flakiness report

Bind results to a plan version

Set BUGMOJO_PLAN_VERSION to a published version id and each result binds to the revision that version pinned, rather than to the case's current revision. That is what makes a run quotable in a release report months later: it names what was executed, not what the case says today.

In CI

Create an API key with write permission in Settings → API keys and store it as a CI secret.
Expose BUGMOJO_API_KEY and BUGMOJO_PROJECT to the test job.
Keep trace, video and screenshot on retain-on-failure / only-on-failure so evidence exists for the runs that need it without paying for it on every green run.
Run the suite as you already do. The run appears under Regression with the commit it was produced from.

Common questions

Can the reporter fail my build?

No. A missing key, no network, an expired token, an oversized trace or a server error each produce one line on stderr and nothing more.

How does a spec find its test case?

A <code>bugmojo</code> annotation, a <code>WEB-42</code> filename prefix, a <code>[WEB-42]</code> title prefix or a <code>@WEB-42</code> tag — whichever fits your repo.

What happens to a failing test with no matching case?

BugMojo creates the case as PROPOSED and files a bug with the spec path and the failure line, so the failure is waiting for you rather than scrolling past in a log.

Is a re-posted run duplicated?

No. Ingest is idempotent on the run and test case, so a reporter retrying its own POST updates the existing result instead of stacking a second one.

Do I need the plan version id?

Only if you want results bound to a published plan version. Without it, results bind to each case's current revision.

Was this helpful?

Updated Sep 19, 2026 · Still stuck? Contact support