What May Sit in the Hot Path

Governing Agents at the Boundary — part 3 of 4

The team is building its own decision point for agent traffic: a small service that a load balancer calls on every request to an agent or tool we own, and which answers allow, deny or something in between. In a component like that, the obvious features are all AI features. Let a model draft the rules. Let a model judge the borderline calls. Let the enforcer understand the request.

Before writing the service, the team wrote down what may run inside it. The result is a short bylaw for the request path. The clauses are not new, and they are the useful part.

The bylaw

Each clause is enforced by a test or a review gate, not by intent.

1. Deterministic and bounded. Every decision has a time and memory budget. For the callout, the measured cost is ten to twenty-five milliseconds, well inside what the load balancer allows. There is no unbounded work, and any outbound call carries a timeout and a declared failure mode.
2. Independent. The enforcer shares no failure mode with what it protects or with any model. The package may not import a model client or an agent framework. That rule is a test: an import allowlist fails the build if one appears.
3. Policy is data, authored off the path. Policy authoring, including authoring by a language model, happens elsewhere. The policy reaches the enforcer as a versioned, signed artifact with an expiry. The LLM writes the policy; it never enforces it.
4. AI judges advise; a deterministic composer decides. A model’s judgment on a borderline call comes from a separate service behind a narrow client, with its own budget and failure mode. Its output is one input to a deterministic composer, never the verdict itself. Hard stops stay deterministic.
5. Safe states are declared in advance. Each failure maps to a written state, and there are no ad-hoc fallbacks in code.
6. Change control scales with the path. Code and policy on the request path go through the stricter gate, and each environment moves from observe to enforce separately. The enforcer’s image is kept small because it is what gets certified.
7. Every decision is recorded, including denials, faults and fail-open passes.

Clause three is the one that matters most for agents. Prompt injection can reach anything that reads the prompt. An enforcer that reads only signed policy and request metadata has no prompt to be injected through.

The safe-state table

Clause five is the one that gets skipped in practice, so the table is written out:

• Stale inputs. If the registry snapshot or the rule set cannot be refreshed, decisions continue on the last good copy for a grace window of about fifteen minutes, and each decision is marked stale. After the window, the enforcer fails closed.
• Cannot decide. If the grace window has passed or there is an internal error, the answer is could not decide, sent as a distinct, retryable status. A policy refusal and a broken decider should never look the same on the wire.
• Decider unreachable. When enforcing, the load balancer rejects the request. Passing traffic through unchecked is permitted only in observe mode.
• Side effects fail. If signing the decision or writing the record fails, the decision is still enforced and the failure is reported loudly. Bookkeeping never blocks traffic.

One more line follows from the table. A decider that fails closed must not cold-start, because a cold start becomes an outage on the request path. Keeping one instance warm is part of the failure mode, not a cost setting.

Why advisory matters, from the field

A managed gateway in front of the reference agents carries a policy judge: a model that reads an English rule and the agent’s next action, and refuses the action if it breaks the rule. It works well. In a live run it refused a single search while letting the rest of the answer through.

It also showed both reasons the bylaw keeps such a judge advisory. In rehearsal it refused a legitimate hand-off, and the reason it gave was a dangerous prompt from earlier in the same conversation, not the act in front of it. In one production project the network path between the gateway and the judge was never built. The judge timed out at thirty seconds and every model call was denied. The response was to detach the judge alone, keep the other two controls enforcing, and split the deployment so that a later mode switch could not quietly bring the detached piece back.

A model in the request path brings its latency, its failure modes and its reasoning into every request. As an advisor behind a budget, it adds signal. Given the final decision, it is a single point of failure that can also be argued with.

Three industries got there first

The team did not invent any of this. Three fields with nothing else in common arrived at the same shape separately.

Telecom separated the policy decision function from the enforcement function a generation ago. The decision side writes rules off the path. The enforcement side sits in the traffic and applies them deterministically, and it has a default rule for when the decider cannot be reached. Routers draw the same line between the control plane and the fast path.

Industrial safety requires the safety system to be independent of the control system it protects, so the two share no failure modes. The functional-safety standards ask it to be deliberately simple, deterministic and bounded by its scan cycle, and to fail to a safe state defined in advance. The paths between zones are declared, and a historian records everything.

Healthcare gives clinical decision support an advisory role: the system advises and the accountable clinician decides. Hard stops, such as a deterministic drug-interaction rule, are kept separate from soft alerts. The rigor required of medical software scales with the harm it can do, and adaptive software needs its changes planned in advance. The audit trail is mandatory.

Each field reached this because an in-path judgment that can be argued with costs the same everywhere. What none of them needed to say is the part specific to AI. A model’s output can be an input to the decision, but never the decision. A model can author policy, but never enforce it.

These are field notes, not settled doctrine. The bylaw governs one component, written while that component’s skeleton was being built. Whether it generalizes is the next thing to test.

The line that stays

Keep deliberation off the request path and enforcement on it. Whatever sits in the path should be simple enough to certify and independent of everything it protects.

Prompt injection can reach anything that reads the prompt. It cannot reach an enforcer that only reads signed policy.