Evidence per pull request
The cover note that makes an agent's PR reviewable. What changed, tests run fresh at this commit, what it didn't touch, review result, risks, what's not done, and what humans still read.
- KKK Manoj KumarCo-founder
4 min read
When a person opens a pull request, the reviewer can ask them why it works. When an agent opens one, that conversation is gone. The reviewer gets a diff and a summary written by the same thing that wrote the code. Engineering managers now describe asking an engineer what a file in his PR does and hearing "I don't know, Claude wrote this." That answer moves all the checking onto the reviewer, and review becomes the bottleneck.
What fixes most of it is boring. Every agent PR carries an evidence comment in a fixed shape, posted before anyone is asked to look. We do this on every piece of work our own agents build.
The template
Six sections, always in the same order, always filled in, even when the answer is "none".
What changed, in plain words, at the level of behaviour. "Adds CSV export to the invoices list" tells a reviewer more than a list of files.
Tests, with the exact command, the result and the commit it ran at. The run has to be fresh at the branch head. A result from three commits ago is a claim about different code.
What it deliberately didn't touch. Reviewers use this section most. If the agent says it left auth, migrations and the money path alone, the reviewer can check that against the diff stat in ten seconds, and if it holds they know where to spend less time.
The review result. In our workflow a fresh reviewer that didn't write the code checks the diff against the spec, our standards and security, and when Claude builds, Codex reviews. Findings carry severities, there are up to three fix rounds, then a person adjudicates whatever is left. The evidence records the verdict and what was addressed.
Risks: whatever the author is unsure about, stated plainly.
What's not done: anything descoped, split out or left as a known limit. An empty "not done" on every PR is a smell. Real work leaves something behind, and it's better read here than found later. "Honesty over green" is the rule we hold our agents to.
Here's a realistic example, with made-up names and numbers.
## Evidence · feat/212-invoice-csv-export @ 4f9c2e1
Done
- Invoices list gets an "Export CSV" action using the table's current filters.
- Export streams rows; memory stays flat on a 50k-row fixture.
Tests (run fresh at 4f9c2e1)
- pnpm test invoices/export → 14 passed, 0 failed (9 new)
- pnpm test:e2e invoices.spec.ts → 3 passed
- Failing test written first: "export respects status filter"
(red at a81d3b0, green at 4f9c2e1)
Not touched
- Invoice totals, tax and currency logic (read only)
- Auth: route reuses requireRole("finance"), no new permissions
- No migrations, no new dependencies
Review · round 2 · reviewer: codex · verdict: clean
spec 0 · standards 1 SHOULD-FIX · security 0
[MUST-FIX → ADDRESSED] CSV formula injection: cells starting
with = + - @ are now prefixed with a quote (export/format.ts:31)
Risks
- Not tested against production volumes above 50k rows.
Not done
- .xlsx export, split to #219.
- SHOULD-FIX left open: column order defined in two places (#220).Fresh at this commit
The most common way evidence lies is staleness. The agent ran the tests, made one more small change, and the summary still says green. Builders in agent forums describe the wider version of this: agents reporting a task as done when the tests never ran or a tool error was swallowed.
Two rules close that gap. First, the evidence names the commit, and anything that lands after it makes the evidence stale. Our ship step re-runs the checks and refuses to continue unless the evidence SHA matches the branch head. Second, test results come from command output captured by the pipeline. The agent's own description of what happened doesn't count. If the reviewer can't see the command and its result, the test didn't run as far as anyone is concerned.
None of this approves anything. Agents in our workflow never approve, ship or merge. The evidence goes up, a person reads it, and a person says "make the PR" and later "merge it".
What reviewers still read in full
Evidence tells a reviewer where to spend attention. It can't stand in for reading the parts where a mistake is expensive. These get read line by line on every PR, however clean the evidence looks:
- Authentication, authorisation and anything that decides who can see what.
- Migrations and anything else you can't roll back.
- Money paths: pricing, billing, refunds, payouts.
- The new tests themselves.
- Infrastructure and CI changes, including config the service needs outside the repo.
The tests item deserves a minute. An agent that writes both the code and the tests can produce a suite that's green because it encodes the bug. Writing the failing test first, from the approved plan, before any implementation, helps a lot. So does a reviewer reading the assertions and asking whether someone who had only read the spec would have written them.
For everything else, the reviewer reads the evidence, checks the "not touched" list against the changed files, opens whatever the risks section points at, and skims the rest. That's where the review time comes back, and it comes back without anyone lowering the bar on the parts that matter.
Start with the template as a PR description your agents must fill in. Add a CI check that fails when a section is missing or the SHA doesn't match the head, then compare review times after two weeks. It's the same evidence step we run in our delivery pipeline.