A green pipeline is a claim, not a fact. It says “nothing we check has failed”. It says nothing about what we do not check.
Working with coding models makes this sharper. A model will happily produce code that passes the tests it was shown and fails the behaviour it was not shown, so the gate stays green while the feature is broken, and nobody notices until a person uses it.
The catalogue
So we keep a document with one entry per class of failure that lint, tests and static analysis cannot see, and for each one, the test that now catches it. It has fourteen entries at the time of writing. Two of them, in general terms, plus one from the rebuild of this website:
- A hook registered on a framework filter that also fires for a second caller the author did not think of. The unit test exercised the first caller only. The fix was a test per route into the hook. (This site’s rebuild log, 2026-10-04: the second-factor filter also ran for API sign-ins.)
- A plan that referenced a file path and a line number that had moved since the plan was written. The code it produced was correct for the wrong file.
- A remembered claim about the deploy state that the deploy log contradicted.
None of these is exotic. Each one passed every automatic gate we had.
What a gate is for
A gate is for the failures you have already had. The catalogue is the memory of them. Every new entry adds a check, and the check runs with the rest: unit tests before every push, static analysis with a baseline that may only shrink, a visual audit against the design tokens.
The second half is a verification layer around the tooling’s own memory. A remembered claim about the code is checked against the git history and the deploy log before a session trusts it; a plan’s anchors are checked before it runs. The model is allowed to remember. It is not allowed to be sure.
The rule we took from it
Every hook, every job, every deploy step gets a line in the plan that names each route into it and which routes it must ignore. The test is written from that line. That is a small discipline, and it is the one that turned “green” back into something we can act on.