I Ran a 105-Agent Audit on My Own Infrastructure

I run a lot of infrastructure for one person. Coquina, the Forge runtime, dozens of workers, an overnight chain, LaunchAgents, Docker services, a homelab full of things that touch each other. The more it grew, the more I had the nagging sense that I'd lost the ability to see all of it at once. I couldn't hold the whole attack surface in my head anymore.

So I did the thing I'd recommend to anyone running a system they can't fully see: I audited it. But I didn't audit it by hand. I pointed roughly 105 agents at it and let them fan out.

Why 105 Agents Instead of One

A single agent auditing a stack this size has a coverage problem. It has a finite context window and a tendency to go deep on the first interesting thing it finds and never come back for the rest. You get a thorough review of one corner and silence everywhere else.

Fan-out solves coverage. Roughly 105 agents, each scoped to a slice of the stack — this service, that worker, this config directory, that dependency surface — running read-only. Read-only is non-negotiable: an audit that can change things is no longer an audit, it's a liability. The agents could look at everything and touch nothing.

But fan-out creates a different problem. More agents means more findings, and more findings means more noise. A hundred agents will happily generate a hundred plausible-sounding issues, a large fraction of which are wrong, duplicated, or theoretical. Coverage without precision is just a wall of false positives you'll never read.

The Trick: Adversarial Verification

Here's the part I'm proudest of, because it's the part that makes agent output trustworthy at scale.

Every finding had to survive a verifier before it counted.

When an audit agent raised an issue, a separate verifier agent took the opposite job: try to refute this. Prove it's a false positive. Prove the file isn't actually reachable, the config isn't actually exposed, the dependency isn't actually used in a way that matters. A finding only made it into the final report if it survived that adversarial pass.

This is the structural move. Fan-out gives you coverage; adversarial verification gives you precision. One agent's confident claim means very little. A claim that another agent actively tried to kill and couldn't — that's a finding worth your time. You're not trusting any single model's output; you're trusting the survival of a finding through an attempt to destroy it.

The pipeline, end to end:

fan-out (~105 read-only agents, scoped by slice)
  → raw findings
  → adversarial verify (refute each, drop what dies)
  → dedup
  → triage into remediation backlog

The Result: 165 Findings, Triaged

After verification and dedup, I had 165 confirmed findings. The severity breakdown was the genuinely reassuring part: zero critical, two high, and the rest medium and low. For a solo-built stack that grew organically over a year, zero critical is a number I'll take.

But 165 findings is also a trap. The failure mode of every audit — human or agent — is that it ends as a 165-item wall of text that you skim once, feel briefly bad about, and never act on. A finding you don't triage is a finding you didn't really make.

So the last step wasn't "produce a report." It was "produce a backlog." Every confirmed finding got triaged into a dedicated remediation board — owned, severity-ranked, actionable. The two highs go first. The mediums get scheduled. The lows get a "someday" lane. The output of the audit isn't a document; it's a queue of work I can actually burn down.

That's the difference between an audit that makes you anxious and an audit that makes you safer. The deliverable of a good audit is prioritized work, not a list of problems.

Trend Tracking: What Regressed

This wasn't the first time I'd run an audit like this, which means I could do something more useful than count findings in isolation: compare against the prior run. What's new? What did I fix that stayed fixed? What regressed?

Trend tracking turns a point-in-time audit into a signal over time. A finding that shows up, gets fixed, and never comes back is a closed loop. A finding that gets fixed and reappears is telling you something structural — that the way I work reintroduces the same class of problem, and the fix needs to be a guardrail, not a one-time patch. You only see that pattern if you diff audits against each other instead of treating each one as a fresh start.

The Humbling Part

The top finding was my own API key, sitting exposed in a dotfile.

Not a subtle one. Not a clever supply-chain thing buried six dependencies deep. A key. In a config file. In plain text. Put there by me, presumably at some point where I told myself I'd clean it up later and then didn't.

I want to be honest about this because it's the whole point. The reason you audit yourself is that you are exactly the person who can't see your own blind spots — you put them there, you've walked past them a hundred times, and they've become invisible through familiarity. The exposed key wasn't a failure of the audit. It was the audit working. The agents had no history with that file, no story about why it was fine, no "I'll get to it." They just looked, and there it was.

Auditing yourself is humbling, and the humbling is the value. If the scariest finding had been something exotic, I'd have learned less. The scariest finding being something basic and self-inflicted is the lesson: the threats to a system you built are mostly the ones you stopped seeing.

What This Is Really About

Strip away the homelab specifics and this is a method for making agent output trustworthy at scale, which is one of the central problems in actually shipping AI systems.

You can't just fan out a hundred agents and believe the results — you'll drown in false positives. You can't run a single careful agent and trust it covered everything — it didn't. The answer is the combination: fan-out for coverage, adversarial verification for precision, dedup for sanity, triage for action, and trend tracking for direction. Each piece compensates for a weakness in the others. Together they turn a swarm of confident-but-unreliable agents into a report you can actually act on.

That's the systems-thinking version of QA, and it scales to far more than auditing a homelab. Any time you're using many agents to cover a large surface and you need the output to be trustworthy, this shape holds: cover broadly, verify adversarially, consolidate, and end with a backlog instead of a wall.

The two highs are already fixed. The key is rotated. And the next audit will tell me whether any of it came back.

If you're building multi-agent systems and wrestling with how to trust their output, that's exactly the kind of problem I like working on: reed@grainlabs.io.