A Record for Every Attempt

Governing Agents at the Boundary — part 4 of 4

The reference deployment first ran in a production project in observe mode. The run finished with zero errors. Every act the deployment exists to show, including a tool outside the allow-list, a dangerous prompt and a search against an English-language rule, went through. Each one also left a record saying it would have been denied. The policy judge had also evaluated calls nobody was watching, the orchestrator’s own tool calls, and recorded verdicts on those as well.

Then the same project was switched to enforce, and the same acts were refused.

That run is the case for this post. In observe mode the decision and its effect differ. The verdict was deny, the effect was allow, and the gap between them is the evidence you need before you turn enforcement on.

Decided is not enforced

A decision log usually keeps one field for the outcome. That works until the same system runs in two modes. In observe mode the decision point says deny, the traffic passes, and a log with one outcome field has to choose which of the two to record. Either choice loses information.

So the record keeps them apart:

• verdict: what the decision point decided;
• effect: what actually happened to the request;
• mode: observe or enforce;
• fail-open: whether the request passed because the decider could not be reached.

Observe mode records deny with effect allow. The “would have” is the point.

The same split covers the cases that are hardest to see. A request that passed because the decider timed out did not pass because it was allowed, and most logs cannot tell those two apart. A request the decision point could not decide is a fault, not a refusal. Each of them is an attempt, so each gets a record.

What the record has to survive

A few properties, each learned from something that went wrong nearby:

• One record per attempt. That includes denials, faults and fail-open passes, not only the requests that reached a backend. A refusal that leaves no row did not happen, as far as anyone can later show.
• A closed, numeric cause taxonomy. Free-text reasons drift and cannot be counted. The draft gives each class its own numeric range: allow, audit, policy denial, decision-point fault, upstream failure. The set is versioned, and nothing outside it is valid.
• Immutable records. A correction is a new record that points at the one it corrects. Signing is planned, with a key version and a link to the previous record so that gaps can be detected.
• Enrichment never mutates the raw record. Joining in cost, ownership or risk happens later, in a separate stage, on a copy.

One more reason the record has to belong to the decision point: the load balancer’s own logs do not carry the client’s workload identity. For an identity-based decision, the decision record is the only witness that says who asked.

The kind of record matters too

This was learned on the managed side. The policy judge’s log holds two kinds of entries: verdicts, and the engine’s own status heartbeats. Heartbeats were about ninety-three percent of the volume. A dashboard that filtered only by log name showed a busy judge on an idle system. One added clause, selecting only entries that carry a verdict, fixed the scorecard, the chart and the log panel. Most of what the judge wrote down was I’m still here.

A record type, stated in the record, would have made that filter impossible to get wrong.

The telephone network already solved this

The shape the team is building toward is borrowed from the telephone switch. A switch writes a call detail record for every call attempt, answered or not. Each record names:

• the calling and called party, and where relevant the party on whose behalf the call was made;
• the recording entity, meaning which switch wrote the record;
• a correlation identifier that survives every hop, so one call can be followed across several switches;
• a release cause from a closed, standard numeric set;
• partial records for long calls, written at intervals and closed by a final one.

Rating, which decides what a call costs, happens later in a separate mediation stage that never touches the raw record.

The mapping to agent traffic is close. The calling party is the agent or user making the call. The called party is the destination, with its owner class: ours, the platform’s, or a third party’s. The recording entity is the instance of the decision point that decided. The correlation identifier is the trace that follows a request from the orchestrator through each specialist. Partial records fit long-lived agent and tool streams, which can stay open far longer than a single request. Mediation covers cost, enrichment and analytics, downstream and separate.

This is not finished. The decision point now runs in two environments and writes one structured record per attempt, as a log line. That line already carries the verdict, the effect, the mode, a numeric cause, the calling and called party, the instance that decided and the trace, and a record that cannot be written is counted and reported, never dropped silently. Not built yet: a table for the records, signing, and partial records. The record still has to be reconciled with the event-log shape the rest of the platform already uses, and several questions are open: the exact cause-code ranges, how long a partial-record interval should be, and whether the client address is stored raw or hashed.

What is settled is the principle. Every attempt gets a record, verdict and effect are recorded separately, causes come from a closed set, raw records are immutable, and enrichment happens elsewhere.

The line that stays

A log of what happened is not enough when the system runs in more than one mode. Record what was decided and what took effect, for every attempt, including the ones that never reached a backend.

Observe mode records deny with effect allow. The “would have” is the point.

The next series turns the same standard on the team’s own work: the places where our green lights were wrong.