Reporting a test result

A ticket comes back from testing and the comment says "doesn't work." Now somebody has to reproduce it from scratch to learn anything, and the round trip costs more than the test did.

A verification comment has one job: let the next person act without asking you a question. That holds whether the tester is a QA engineer, the developer who wrote the feature, or an agent running an acceptance suite.

What the comment has to contain

Start with the verdict. Pass or fail, on the first line, before anything else. Someone scanning twenty tickets should not have to read a paragraph to learn which ones need attention.

Say where you tested. Environment, URL, account, and any configuration that differs from the default. A result nobody can reproduce is not a result.

List what you checked, one line each, with its own verdict. Not "tested the checkout flow" but the individual cases and how each one went. A partial pass is information; "tested the checkout flow" is not.

For a failure, say what happened and what you expected. Numbered, one failure per line. Attach the screenshot, the recording or the stack trace. A screenshot of the broken state answers questions you did not think to ask.

Keep concerns separate from failures. A slow page that still works is not a failed test. Note it, tag whoever owns the decision, and leave the verdict alone. Mixing the two trains people to argue with your verdicts.

A comment that does all of that:

🚫 Failed

Tested on: staging, stg.example.com, account qa-04@example.com, currency GBP

Results:

1. Log in with a valid password — pass
2. Log in with a wrong password — pass
3. Add an item to the cart — pass
4. Submit the order — fail. The Submit button posts nothing and the console shows a 500 from /api/orders. Expected the order confirmation. Recording attached.

Note: the cart page takes about 6 seconds to paint on a cold load. Not a failure, flagging for @pm.

The same rules apply to agents

An agent that runs an acceptance suite is producing a verification comment, and the failure output most suites emit by default is the "doesn't work" version: a stack trace, no environment, no list of what else passed.

Make the agent report the same shape. It has an advantage here, because it already knows the environment, the account, the seed data and the full list of cases it ran. It only has to say so.

The payoff is that a human reading the comment can decide whether to reproduce it without opening CI, and the next agent picking up the ticket has the reproduction steps in the ticket rather than in a log that expires in thirty days.

Settle on one format

Whatever shape you land on, write it down and reuse it. A template removes the decision from every individual report, which is what makes the reports consistent enough to skim. It also gives an agent something to fill in rather than something to invent.

Revise it when it stops fitting. A format that survives contact with a few hundred tickets and never changes is usually one nobody reads.