4 minute read

[novel-gate] SKIPPED

The gate said the lane was off. The configuration said the lane was off. In the same scan, the lane-summary log attributed 48 findings to that lane, and the scans were hitting a rate-limit error that looked exactly like the expensive thing the gate existed to stop.

For days, that is what the cloud scans showed. Three sources of truth, and one of them disagreed.

The first reading

The scanner finds vulnerabilities with two lanes. One is deterministic: established static-analysis tools, cheap and predictable. The other is agentic. It used to walk the repository file by file with a model, which is thorough and very expensive. We had built a cheaper structure to replace that walk and put the old lane behind a flag, off.

So when the log kept reporting findings from the agentic lane, and the scans kept running into quota errors, the obvious reading was: the old walk was still running somewhere, the flag wasn’t holding, and it was burning the quota on every scan.

We acted on that reading before we had proved it. The flag’s default was inverted so that its absence meant off, everywhere. The gate got its own log line and a typed read of its configuration, so it could say for itself whether it had run. It said SKIPPED. The log still said the lane had produced findings.

How both were true at once

The two lanes are merged before the scan reports. The merge function took the agentic lane’s results and the deterministic lane’s results, and it reconciled them in place: it appended the deterministic findings into the agentic lane’s own list and returned it.

The lane-summary log ran after the merge. It counted the agentic lane’s list. By then that list held the merged total.

The agentic lane had emitted zero findings the whole time. It was dark exactly as configured. The 48 was every finding from both lanes, attributed to the lane that had produced none of them. The gate was right, the configuration was right, and the log was reporting, accurately, the size of a list that something else had changed under it.

The rate-limit errors had a separate explanation. One scan was rerun on fresh quota and completed clean, so the error from the night before had been quota already drained, not a runaway walk. The real heavy consumer turned out to be a third lane, one that enriches findings after the merge rather than discovering them. It was carrying about 88% of the scan’s token cost. Because it runs after discovery, gating it loses no findings, and it got a flag of its own.

What changed

The log reads what it means. The lane-summary line now snapshots each lane’s raw count before the merge and logs those. It reports what each lane produced, not what a shared list contained at the moment the log line happened to run.

The merge cannot do it again. The log fix treated the symptom. The cause was an impure function in a hot path, so the merge was rewritten to be pure by construction. It builds a new result from its inputs: corroborated findings are fresh copies, additions are copies, rolled-up fields are new lists. Neither input is touched.

There is a test that fails if it ever mutates again. It merges a one-finding agentic lane with a two-finding deterministic lane, checks that the result holds both, and then checks the inputs: the agentic lane still has one finding, its finding is still attributed to the agent alone, its provenance, notes and errors are all exactly as they went in, and the deterministic lane is unchanged. Whoever reads either input after the merge, whether a log line, a metric or the next stage, gets what that lane actually produced.

The class, not the incident

A log line feels like a primary source. It is a statement printed by the system, at the time, about itself. But a log reads a value at the moment it runs, and the value may already have been changed by something between the operation and the print. The log is downstream of the data.

When a derived signal (a log line, a metric, a dashboard count) contradicts an authoritative one (a gate decision, a configuration flag), the derived signal is the weaker witness. Trust the authority and then hunt for whatever mutated the thing the derived signal reads. The hunt usually ends at an in-place update somebody wrote for convenience, long before anyone logged around it.

The false reading pointed at the most expensive component in the system, which was already switched off, and the first response was to reinforce a gate that had held from the start.

The line that stays

When the log disagrees with the gate, don’t first ask whether the gate is broken. Ask what touched the value between the decision and the print.

A log is downstream of the data. If something mutated the data first, the log is a lie.