I run a publishing gate that won't let a page go live unless every number on it traces back to a declared piece of evidence. A draft carries an editorial.json file listing evidence entries, each tagged evidence:, and one of five machine gates, numeralgate, strips a page's HTML down to plain text, pulls every number out of that text, and blocks the draft if even one number has no matching entry. All five gates have to pass before a human ever sees the draft to sign off on it, and the verdict is permanent per draft — edit the body, and you start over from a clean dossier.
I hit the edge of that design building a payment-fee calculator page with a separate site-builder tool I run: type in a revenue figure, get back the fee and the amount you actually take home. A calculator like that needs JavaScript, and the JavaScript needs to know the fee rates. The gate, though, cannot read inside a tag. Whatever turns a page's HTML into checkable text either drags the script body along as text — a false positive, flagging numbers that were never really claims — or drops it entirely, a false negative on exactly the thing the gate exists to catch.
The first is to teach the gate to parse JavaScript. That means putting a JS parser inside the gate, which means the gate now owns a second language runtime — every bundler, minifier, and syntax change downstream becomes its maintenance debt from then on. Worse, it's a fight where the losing condition belongs to whoever edits the code next: twist the JavaScript slightly and the parser loses.
The second is to exempt tags from the check entirely. That one is more dangerous than it looks, and I know because I'd closed a version of exactly this mistake earlier the same morning. numeralgate carries an exemption for alphanumeric identifiers — CSS values like 20px, say — so they don't get flagged as unsourced numbers. British pence notation, 20p, was matching that same exemption and riding straight through the check, untouched. A convenience exemption had quietly become a laundering channel. Exempting wholesale would have reopened that same defect as a much bigger hole.
So I took a third path: instead of teaching the gate to read the JavaScript, I made sure there was nothing in the JavaScript worth reading. The fee rate travels like this:
The rate rides into the page as a data- attribute on a table row. That row is HTML, which the gate can read, and its cell value is already rendered from a cited evidence entry. The JavaScript itself never sees a fee rate — it reads a value that's already sitting on screen, already proven, and does arithmetic on it. The gate ends up able to verify where this calculator's rates come from without interpreting a single line of JavaScript.
A direction like that isn't a guarantee by itself. It only becomes a rule once every part of it has an enforcement point attached, and this one has five. Rates enter only through an evidence reference. validatecalculatorevidence checks that every rate in a calculator row points at a declared entry; write a literal number directly into the row and the build fails. Every numeric literal inside any has to belong to the set {0, 1, 2, 100}. This is a new gate, SCRIPTNUMERICLITERAL, and it's the one actually carrying the weight of the design. It isn't a parser — it's a whitelist. 0, 1, and 2 cover indices and sign; 100 covers percentage division. A domain number structurally cannot fit inside that set. Adding it meant bumping the gate's schema from v4 to v5. Calculator output has to equal the declared derived value. A small test harness runs eight scenarios and checks for an exact string match against what the gate computed as the derived value — rounding, currency symbol, and digit count all have to agree. Script and style content never becomes body text in the first place. extractbodytext, the function that pulls a page's body text for checking, removes both elements wholesale before it ever strips a tag, so there's nothing left for the gate — or anything downstream of it — to accidentally read out of a script block. The fact-check sheet a human signs shows the evidence the calculator cites. Whoever signs off sees the rate's provenance on the same screen as the number it produced.
There's a coda to that fourth invariant worth telling on its own. extractbodytext already had a docstring warning about exactly this situation, before the calculator page existed. It said, in effect: content inside and isn't visible on screen, but this function drags it along as text anyway; since nothing in this package puts either one into a page body yet, that's fine for now, but the day it does, fix this function first.
That day arrived. The calculator was the first page this package had ever put a into.
The lesson isn't that the comment was well written. A comment that names its own precondition is a tripwire — but a tripwire only works if someone actually walks over it. That docstring sat there doing nothing for months, and it would have done nothing this time either if I hadn't happened to read that function. Without code that detects the precondition breaking, a comment like that isn't an alarm. It's a note left behind, read only after the fact.
The third invariant's harness needs a JavaScript runtime to execute its eight scenarios. As first written, if that runtime wasn't on the machine, the test called pytest.skip.
That's not a check. On any machine that's missing the runtime — including CI — the check just quietly disappears, and the color stays green. There's no way, from the outside, to tell "this passed" apart from "this never ran."
I changed it to pytest.fail. No runtime now means a red light, and someone has to either install it or make a conscious decision to turn the check off — silence stops being an option. One older test in the same file kept its skip, with a docstring explaining why that one is different. To confirm the fix actually worked, I pointed the process's PATH at an empty directory to genuinely hide the runtime, and watched the test report FAILED.
The gate also did something I hadn't asked it to do. I decided to cut a numeric claim from the body text — a line about a $14.99 item — and just deleted the sentence. warrantgate blocked the draft anyway: evidence was still attached to that claim, and now nothing referenced it. An evidence item had gone orphaned.
The tempting fix was to keep the evidence alive by hanging a footnote off some other sentence in the prose. Instead I checked the code first. editorial.sections[].html gets rendered exactly as written, and the only path that attaches an evidence marker runs through structured calculator and table cells — never through prose. There was no way to fake having one. So I deleted the claim and its evidence together, and the evidence count dropped from 13 to 12. The gate pushed me toward the honest fix instead of the convenient one.
None of this tells you whether the new gate actually catches anything. Passing it proves nothing on its own — you have to break it on purpose.
So I did, twice. I injected the literal rate 3.49 straight into the JavaScript, and SCRIPTNUMERICLITERAL blocked it. Then I deleted the script-removal logic from extractbodytext, and the corresponding test went red. Both times I reproduced the failure myself rather than trusting a worker's self-report that it worked.
When there's a region a verifier can't look inside — a JS bundle, a binary, generated code, a third-party embed — the fix isn't to extend the verifier's reach into that region. It's to restructure things so that region never holds a claim that needs verifying in the first place. Move the claims onto a surface you can verify, and let the opaque region be nothing more than a pure function of that surface.