Nick Perkins

Software Engineer. Volunteer Motorsport Official. ADHD Brain. Bit of a nerd.

Running Agents in Parallel Only Works With a Review Gate

Last night I pointed a coding agent at 13 open issues on the food inventory app I built for the house, and told it to manage the work and ask me only when it needed a human. About seven hours later I hit my weekly usage cap, with nine PRs merged and three still in draft. Not bad for a night’s work.

Launching agents in parallel is the easy part. Trusting what comes back isn’t.

Triage first, so the agents had a spec

Before any code, I ran the triage skill from Matt Pocock’s skills over the issue list. It filled the gaps in the labels, wrote agent briefs for the two issues that still lacked one, and added concrete numbers to three more. It also split my big multi-factor auth issue into four, so each fit in one agent session. Every implementer was told the issue body and the brief comments were its spec, numbers included.

You wouldn’t accept a half-scoped ticket as ready for development. An agent will, and it will do what every junior developer does with that sort of ticket. A poor job.

Then I asked it what order to do things in, and it started launching agents. Each one got its own git worktree and branch, so four of them could edit the same repo without standing on each other.

I added the review rule a minute in

The first wave had barely started when I told the agent to run an adversarial review on every task and fix what it found. I’ve done this before, and I know what happens when there’s no review. That one sentence shaped the whole night.

A review agent got the branch and the issue and was told to assume the author was wrong and try to prove it. It was read-only. It could run scripts and write to a scratch folder, but it couldn’t edit, commit or push. In total the agent launched 12 implementers and 10 reviewers.

It feels like you’re wasting time and tokens by running almost double the amount of subagents, but it’s really worth it in my view.

The reviews found what the passing tests couldn’t

One issue was an external monitor for the app: a small script on a timer that checks the health endpoint and the certificate expiry, and mails me when something is wrong. The implementer’s version came back with 40 passing tests. The reviewer found it would fail on every run once installed.

The systemd unit ran as a dynamic user, and the env file was readable only by root. The script tried to read it, got permission denied, and exited before checking anything. Every run would have mailed me a failure. The tests missed it because they ran as whoever invoked them, who could read the file.

There’s nothing like monitoring that doesn’t actually do anything.

That’s the gap. Tests tell you the code does what the tests expect. The tests written by the same agent that implemented the code. A reviewer who’s been told the author is wrong goes looking for the gaps in those tests, like a QA engineer would.

The cross-tenant test was the more interesting one. It’s meant to fail if any API route forgets to scope its query to the household. The reviewer broke the code about two dozen different ways and checked which breaks the test noticed.

Most were caught. Among the ones that got through were a catch-all route registered ahead of the error handler, and a route that trusted a household ID from the query string.

The sign-up abuse controls had the worst finding. One account could mail any address with no limit. Worse, someone could block every user’s password reset and sign-up without ever solving the bot challenge. It’s pretty basic stuff, and any engineer worth their salt wouldn’t let that ship.

A shared test database broke the parallel runs

All the agents shared one test database. One branch ran its migration against it and left a column behind, and a schema test failed on every other branch for a reason that had nothing to do with them.

The agents already took a lock before touching the database, so two never ran at once. A lock stops two things using it together. It doesn’t clean up after the first.

The fix was recreating the database. The handoff note now says never to run two branches’ migrations against the same one.

The reviewers had a gap too. The security reviewer had no shell, so it could read code and reason about it but couldn’t run the tests. Its findings need a follow-up pass where something executes them.

Unreviewed work stays out

The cap hit with three branches unfinished. The agent stopped, pushed them as draft PRs, and wrote a handoff note with the merge order and the state of each. Nothing was lost.

The release I tagged afterwards came from main only. Two of the three drafts had never been reviewed, so they stayed out. That’s the rule I’d keep above everything else: unreviewed work doesn’t merge, even when it’s nearly done, and even when you’d like it shipped tonight.

A few things were never going to be agent work. Creating the bot-challenge widget and setting its keys, running commands with sudo on the server, and following the key-rotation runbook once to see if it works. The agent listed them for me instead of guessing.

Add the reviewer before the second agent

I wrote last year that AI makes you faster at being the engineer you already are. Agents do the same thing to your process. Without a review gate they merge bad work faster, and the passing tests make it look fine.

If you’re about to run more than one agent at a time, set up the reviewer first. Learn from my mistakes, so you don’t have to. Two of my drafts are still waiting on the cap to reset, and neither merges until one has tried to break it.

Thanks for reading. I'm Nick, and this is my personal blog. If you got something out of this, there's more where it came from.

Browse the archive →