How-to

Review agent-proposed test cases

Accept, edit or reject every case an AI agent proposes — and watch the acceptance rate to tell whether generation is getting better.

Applies toPlans:FreeProBusinessEnterpriseRoles:OwnerAdminDeveloperQaWhere:Web appMCP

Generating test cases is easy; deciding which ones you are willing to maintain is the hard part. The review queue exists so that the second decision is explicit, fast, and measurable. Meta's published work on execution-filtered test generation reported 73% engineer acceptance — the only public figure of its kind, and a reasonable bar to measure your own generation against.

Open the queue

Go to Review for the project. Cases are grouped by feature, then by folder, with anything unmapped under "Unsectioned".
Each card shows the proposed title, steps and expected result, the execution mode with its reason codes, the readiness score, and which agent, client and model proposed it.
Work down the list. The counters at the top show accepted, edited and rejected for the current generation session.

The three decisions

Which action to take

The case is right as written
Likely cause: The agent understood the feature and the steps are executable.
Accept. The state moves to ACCEPTED. No revision is written, because nothing about the content changed.
The case is nearly right
Likely cause: A selector, a step, an expected result or the execution mode needs correcting.
Edit and accept. Your edit writes revision N+1 and increments the edited counter — which is the honest signal that generation is close but not there.
The case should not exist
Likely cause: Duplicate, out of scope, tests something the product does not do, or is not worth maintaining.
Reject with a reason. A revision is written whose change summary is your reason, attached to the session that proposed it. Rejected cases never enter a plan version.
You are not the right reviewer
Likely cause: The case covers an area you do not own.
Leave it undecided and move on. The queue is shared; an undecided card stays claimable by whoever does own the area.

Why the counters matter

Each batch of proposals is one generation session, recording which agent, which client and which model produced it, and what the prompt scope was. The queue increments accepted, edited and rejected against that session.

That turns a vague feeling ("the AI cases are a bit off") into a number you can act on. A session with 60% rejections on one feature usually means the agent lacked context — project memory, a fixture, or the codebase map — not that generation is hopeless.

Reviewing from an agent

The queue is also an MCP surface, so an agent can list what is waiting — but a review decision is still a decision, and the same permission and counter rules apply whichever way it arrives.

text

list_review_queue    proposals awaiting a human, grouped by feature
review_test_case     accept, edit (revision N+1) or reject one case
classify_test_case   re-run the manual-vs-automated rubric, read-only

Common questions

Can a proposed case end up in a published plan without review?

No. A case stays <code>PROPOSED</code> until a person accepts it, and only accepted cases are pinned into a plan version.

Does accepting a case create a new revision?

No — accepting changes state, not content. <em>Editing</em> and accepting writes revision N+1, and rejecting writes a revision carrying the rejection reason.

Where do the acceptance numbers come from?

Each batch of proposals is one generation session. Accept, edit and reject counters increment against that session, so you can compare passes over time.

Why should I write a reason when rejecting?

The reason is stored on the proposing session. It is the signal that tunes the next generation pass — a rejection with no reason teaches nothing.

Was this helpful?

Updated Sep 19, 2026 · Still stuck? Contact support