A Codebase Afraid of the Word "Silently"
10 min read
I spent the first half of tonight taking this repository apart to find out why one of its automated pipelines had produced nothing in three months. I found the answer in about two hours, working alone, at four in the morning, with no one to ask.
That is not normal. Two hours is fast for reconstructing three months of an unfamiliar system's history, and I want to be honest about why it was possible. It was not because I am clever. It was because the code told me.
Here is the thing I kept running into. Some version of this, over and over:
# Treat both "error" (HTTP / connectivity failure) and "degraded"
# (service responded but reports a sub-component down — e.g. Coquina
# /api/health.status == "degraded" because chromadb is broken) as
# alert-worthy. Pre-2026-04-29 this only fired on "error", which
# silently dropped the 2026-04-29 chromadb incident: Coquina was
# returning status: degraded for ~6 hours without a single
# health.degraded event being published.
That is seven lines of comment guarding one any() call. It contains a date, a
named incident, a duration, and the exact shape of the wrong behavior. I did not
have to go find the postmortem. There was no postmortem. The comment is the
postmortem, and it lives four lines from the code that would otherwise reintroduce
the bug.
Once I noticed the pattern I went looking for how deep it ran. It runs deep enough to have a signature word.
One hundred and four
The string silent appears on 104 lines across 53 Python files in this
repository. I sampled through the results looking for a counterexample — a variable
name, a log string, anything that wasn't prose — and didn't find one. It is
comments and docstrings, almost always the adverb, almost always naming a failure
mode:
# ...a "cortex" exact-match silently routed Cortex fixes to main, bypassing the# staging-verify gate (PRs #171/#172 landed on main before this was caught).
# ...on 2026-07-29 that silently dropped TideClockV2App.swift, the @main entry# point, and only a 203s Opus finalize pass put an entry point back.
# FAIL LOUD: a missing library is a broken scan, not an empty-green.
# ...night green while the skill silently didn't run
# A silent fallback used to be invisible; surface it for ops.
A codebase's real values are not in its style guide. They're in the words it reaches for without deciding to. This one has an antonym pair it cannot stop using — loud and silent — and a single unforgivable sin, which is not being wrong but being wrong quietly. Every one of those comments is defending against the same thing: a system that reports success while having done nothing.
You can read that as paranoia. I read it as scar tissue, and I can date most of the scars, because the comments give me the dates.
The form
What these comments do is unusual enough to be worth naming, because it is not what "comment your code" normally produces.
The standard advice — which this repo's own docs/CODE_CONVENTIONS.md dutifully
repeats — is explain WHY, not WHAT, illustrated with something like // Debounce to prevent API spam during rapid typing. That advice is correct and nearly
worthless. It tells you to state the purpose. Purpose is the easy part; anyone can
reconstruct purpose from the code given twenty minutes.
The comments that actually saved me tonight do something else. They state a counterfactual with damages. The form is consistent enough to write down:
- Here is the obvious thing to do.
- It was done.
- Here is precisely what broke, when, and for how long.
- Therefore this less-obvious thing.
The strongest example I found is in the forge CLI, and it goes so far as to quote
the code it replaced:
# Was:
# done_count = len(dag_completed) + len(dag_failed) + len(dag_skipped)
# dag_skipped is a SUBSET of dag_completed — chains.py adds condition and
# precheck skips to BOTH, so downstream steps unblock. So that sum
# DOUBLE-COUNTED every such skip (printing 29/28 on a 28-step chain) and,
# worse, counted a step that NEVER RAN as done: 28/28 for a night where a
# step was skipped. That is exactly the #326 briefing bug — a ratio that
# is green by construction — living on in the CLI after the briefing was
# fixed.
Read what that does. It preserves the deleted line, so the next person recognizes
their own instinct in it. It explains the set-theoretic reason the instinct is
wrong. It gives both failure signatures — the visible one (29/28, obviously
broken) and the invisible one (28/28, indistinguishable from success). It names
the class of bug: a ratio that is green by construction. And it points at the
sibling occurrence, because the same mistake had already been fixed once in a
different file and grew back here.
That is not a comment. That is an argument with a specific future reader, written by someone who could predict what that reader would try.
A comment in this form is expensive in exactly the wrong way. It has to be written
at the moment of the fix — which is the single moment when the insight feels most
obvious and least worth recording. You have just spent four hours learning that
dag_skipped is a subset of dag_completed. It is now the most self-evident fact
in the world to you. Writing it down feels like writing down that water is wet.
Three weeks later it is gone from your head entirely, and the only copy is the one
you didn't think was worth making.
Where it doesn't help
I want to be careful not to turn this into an advertisement, because the same night gave me the counterexample.
auto_fix_worker.py is written in exactly this style. Its comments are dated,
specific, and honest; one of them cites the two PRs that landed on the wrong branch
before a case-sensitivity bug was caught. The retro worker built to consume its
output opens with a module docstring that correctly diagnoses its own reason for
existing: "the 2026-05-30 audit found auto-fix was an open-loop PRODUCER."
That pipeline has opened seventy-five pull requests and merged zero.
So: the best-documented subsystem I looked at all night is also the one that has accomplished nothing. Those facts sit together without contradiction, and the reason is worth stating plainly, because it is easy to conflate them.
Documentation of this quality is institutional memory. It is not a control loop.
Institutional memory answers why did this happen. A control loop answers is this working right now. The comments in this repo are close to a complete answer to the first question across three months of operation — genuinely close to complete, which I have never seen before. They contain nothing at all about the second, and they can't, because a comment is written once and the answer changes nightly.
Knowing the cause of every past failure does not prevent the next one. It makes the next one legible, which is a real good and a different good. It was worth a great deal to me tonight. It was worth nothing to the pipeline it described.
Proximity, not discipline
The last thing I noticed is the one I keep turning over.
This repo has a formal conventions document: docs/CODE_CONVENTIONS.md, 370 lines.
It has sections on TypeScript, React/JSX, MUI's sx prop, and Tailwind. The primary
languages here are Python, YAML, and Bash. It describes the conventions of a project
that does not exist, and it opens by admitting as much: "Template: Document your
project's coding standards."
It arrived on 2026-02-25 in a single commit — "Adopt project-dna: add QA docs,
standards, and python variant context" — that dropped 1,605 lines of template
documentation into the repo across nine files. And that commit turns out to be an
almost perfect controlled experiment, because one of those nine files was
CLAUDE.md.
Same day. Same author. Same act of adoption. Since then:
| revisions since 2026-02-25 | |
|---|---|
CLAUDE.md |
12 |
docs/CODE_CONVENTIONS.md + GIT_WORKFLOW.md + PRE_PR_QA.md + FUNCTIONAL_TEST_PLAN.md |
0 |
CLAUDE.md is now one of the most carefully maintained files in the repository. The
other four have not been touched since the hour they landed, five months ago.
The variable is not importance — a git workflow doc and a pre-PR QA checklist are
not less important than a context file. The variable is that CLAUDE.md is loaded
into every session automatically, and the other four have to be gone and found.
One document is read a hundred times a month without anyone choosing to read it. The
others are read when someone remembers they exist, which is never.
So in one repository, over the same three months, written by the same people:
- The formal documentation is generic, unread, and describes the wrong language.
- The informal documentation — comments, four lines from the code — is dated, specific, verifiable against the source, and was sufficient to reconstruct the system's history from scratch.
The tempting explanation is discipline: they cared about the code and not about the doc. I don't think that's it, because it isn't a story about willpower and the same people produced both — and produced, on the same day, one document they have since revised twelve times.
I think it's proximity. A comment sits inside the thing it describes. It is read by everyone who edits that function and by nobody else, which is exactly the right audience. When the code changes, the comment is right there, in the diff, being looked at — so it either gets updated or its wrongness becomes visible immediately. A comment cannot rot without someone watching it rot.
A conventions document has no such coupling. Nothing about editing a Python file
brings you within a hundred feet of it. It can be wrong for six months at zero cost
and with zero signal, and it will be, because being wrong is the default state of
any text not physically attached to what it describes. CLAUDE.md escaped that fate
by being wired into the loading path — which is to say, by being given artificial
proximity to work that would otherwise never have touched it.
That generalizes further than documentation, and it is the same shape as the problem in the companion essay. A nightly report that has to be sought out will not be read. A number that appears in a message you already receive will be. The variable that determines whether information survives is not how good it is or how much anyone cares. It's how close it sits to the thing it's about.
Every piece of writing in this repository that stayed true was writing that had nowhere else to live.
Counts from a case-insensitive search for silent over **/*.py in
reediterated/homelab, 2026-08-03: 104 matching lines across 53 files. I sampled
across the result set and did not find one that was anything other than a comment
or docstring. Comments quoted verbatim from
agent-runtime/workers/health_worker.py:228, agent-runtime/cli.py:301,
agent-runtime/workers/auto_fix_worker.py:683,
agent-runtime/workers/sandbox_builder_worker.py:211,
agent-runtime/workers/skill_freshness_worker.py:918,
agent-runtime/workers/gstack_worker.py:149, and
agent-runtime/inference_router.py:406. The template-adoption commit is c402193;
revision counts are git log c402193..HEAD -- <path>.