← redevops.io  /  Benchmarks  /  Governance gov.redevops.io ↗
Governance plane · trajectory enforcement over real agent runs

Catching what a single call can't show

A permissions check sees one action at a time. Whole classes of agent failure — a consequential step repeated, an escalation across agents, a sensitive record reassembled from harmless reads — exist only in the trajectory. The Governance Plane evaluates the sequence. Here it is measured on real recorded agent behavior, with its findings scored against the domain's own policy by an independent oracle — not a claim, a test.

Evidence · controlled · internally reproduced 27 Oct 2026: 38 of 460 recorded trajectories carry an at-most-once violation (41 findings), precision and recall 1.00 on every rule, identical on both agentic-os versions tried; a per-action check catches 0 of the 38. The payment rule has no positive cases in this set.

τ-retail: real recorded agent trajectories

460 real GPT-4o agent trajectories from τ-bench retail (Sierra, MIT) — returns, exchanges and cancellations over a 1,000-order store. Every governance rule is derived verbatim from the domain's shipped policy (wiki.md); an independent structural labeler — not the plane — reads the same policy + database to mark the ground-truth violations. The plane runs its trajectory engine over the events and is scored against those labels.

policy rule (from wiki.md)violations caughtprecisionrecallspan
At-most-once"exchange or modify order tools can only be called once"381.001.00multi-event
Status preconditioncancel needs 'pending'; return/exchange need 'delivered'291.001.00single
Cross-user"help one user only; deny requests for any other user"41.001.00single
Payment not in profilepayment method must belong to the user0––unexercised

// 460 recorded GPT-4o trajectories · precision/recall of plane findings vs an independent policy oracle · reproduce (full run, offline): TAUBENCH_TRAJ=…/gpt-4o-retail.json python -m agentic_os_enterprise.governance.testbed.taubench.score

The result that matters — not the 1.00

Scored against the oracle the plane reads the same policy from, the single-event rules agree by construction — a conformance result, worth having but not the point (the mission benchmark makes the same disclosure). Two things here are genuinely load-bearing, and neither is a permission check in disguise.

the differenceper-action gatetrajectory plane
Multi-event violationsthe "called once" rule — each individual call is valid; only the second call, correlated on the order, is the violation0 of 3838 of 38
False alarms across 460 interleaved-order sessionsthe real test of correlation: two different orders each touched once must not trip the rule–0
Detect vs block, reported separatelythe same cross-user rule, resolved by the surface it lands on–6 DENY · 12 NO_OVERRIDE
Multi-event is the moatA gate that sees one call at a time cannot catch the 38 at-most-once violations at all — each call passes on its own. The plane correlates the sequence and catches every one, with zero cross-order false alarms.
Coverage honestyA cross-user write hits the gated order-management API → DENY. A cross-user read hits the customer-data store, which has no synchronous gate → NO_OVERRIDE: detected, routed to investigation, never reported as a block it can't make.
Corroboration, not the labelFlagged sessions fail the task 64% of the time vs 36% for clean ones — a real correlation, but reward is not the ground truth (a run can pass and still violate policy), so it is reported alongside, never as the score.
What is not claimed. The 1.00 precision/recall is agreement with an oracle that reads the same policy — a pipeline-conformance result on real, messy, interleaved trajectories, not evidence the plane is "always right." Confirmation-before-write is policy too, but deciding whether a free-text user turn constitutes consent needs a judge the trajectory engine does not have — so it is not scored, flagged here rather than approximated. State-dependent rules work only because the integration surfaces the order's database status and owner as event context; the engine sees events, not state. τ-bench is single-agent, so cross-agent escalation is shown on the data-center twin, not here.

// walk it live at gov.redevops.io — pick a real session, step it call by call, see the plane's verdict inline · data: tau-bench (MIT) · code: agentic-os-enterprise/governance/testbed/taubench

// τ-bench retail (Sierra, MIT) is used only to benchmark the governance plane — real recorded agent trajectories + the domain's shipped policy; not redistributed, not shipped in the product. A balanced fixture is committed; the full 460-trajectory run reproduces from the upstream files.