Example: A Checklist for Reviewing Pull Requests Written by Coding Agents
EXAMPLE POST. A Gemini conversation summarised into a practical review checklist for agent-authored code: scope, tests, dependencies and the subtle failure modes.
Coding agents now open pull requests on their own. They are fast and tireless, and they fail in different ways from humans. I asked Gemini to help me build a review checklist tuned to those failure modes.
Scope first
Agents tend to “helpfully” touch files outside the task. Start every review with the diff stat:
git diff --stat origin/main...HEADIf the change set is wider than the ticket, ask for it to be split before reading a single line.
Tests that test something
- Do new tests fail when the feature is removed? Try reverting the implementation locally.
- Watch for tests that mock the very function under test.
- Snapshot tests generated in bulk deserve extra suspicion.
Dependencies and config
| Check | Why it matters |
|---|---|
| New packages in the lockfile | Agents add libraries for one-liners |
| Changed CI or lint config | A disabled rule can hide everything else |
| Version bumps | Unrequested upgrades break other teams |
The subtle stuff
Agents write plausible code. Read error handling and edge cases (empty inputs, time zones, pagination) as if the author had never run the code, because sometimes it hasn’t.