The top of the stack. Representation planning chooses one access path; mission planning composes several into a governed decision - resolve identity, gather evidence across representations, verify, and abstain or recommend. The test is deliberately not "can the graph database find owners" - everyone knows it can. It is: can the planner compose that evidence into the right decision, for the right reason?
150 onboarding scenarios over the real GLEIF + OpenSanctions graph (182,445 entities in Neo4j), labelled as facts about the data: NO-GO (a sanctioned entity anywhere in the ownership chain), GO (clean chain), ABSTAIN (the name matches more than one entity - identity unresolvable, escalate). The decider composes: resolve identity → traverse ownership → screen every owner for sanctions → verify and recommend.
| 150 scenarios · 50 Go / 50 No-Go / 50 Abstain | Composed decision (the planner) | Naive single-node screen (baseline) |
|---|---|---|
| Decision accuracyGo / No-Go / Abstain vs ground truth | 100% | 36.7% |
| Sanctioned entities onboardeda No-Go waved through as Go - the dangerous error | 0 | 45 of 50 |
| Abstains on ambiguous identityescalates instead of guessing the wrong entity | 50/50 | 0/50 |
| Explanation correctNo-Go cites the exact sanctioned node | 100% | - |
| Decision latencyfull compose + verify + decide | 0.6 ms | - |
The sanction is upstream 45 of the 50 No-Go applicants are clean at the name - the sanctioned entity is an owner, one to eight hops up. A single-representation sanctions lookup sees a clean applicant and onboards it. Only composing the ownership traversal with the screen catches it.
Abstention is a first-class outcome All 50 ambiguous-identity cases are escalated, not guessed. A KYC system that auto-decides on the wrong "Acme" is worse than one that asks. The composed decider abstains; the baseline never does.
Right decision, right reason Every No-Go cites the exact sanctioned node, and the full chain is preserved as replayable evidence. The recommendation is auditable months later, not a black box.
What it is not Not a test of Neo4j (assumed to work) and not a fuzzy human judgement (labels are deterministic facts). It measures the runtime's composition and governance of the decision.
sanctioned flag is derived: a GLEIF entity is flagged when its name matches an real OpenSanctions target name by exact, case-insensitive string match (3,765 of 182,445 entities). So a flag is a real match, not an injected label - but it is an exact-name match, which is deliberately the assumption the matching-robustness half below stress-tests. One consequence: a name collision can flag a legitimate entity (e.g. a "Standard Chartered" subsidiary), which is harmless as a graph-traversal fixture but is why the composition, not the flag, is the thing under test here.
CR-enterprise/benchmarks/representation-bench-kyc/curate_scenarios.py.
// 150 scenarios over real GLEIF (182,445 entities) + OpenSanctions in Neo4j · composed decision vs a naive single-node screen · decision accuracy · abstention · explanation · latency · reproduce: CR-enterprise/benchmarks/representation-bench-kyc/
Fair criticism of the benchmark above: the applicant already arrives as an exact graph node, and the labels are deterministic, so the composed pipeline scores 100% by construction. This second benchmark removes that crutch and tests where the composed decision can genuinely be wrong: matching the applicant to the sanctions list when the name does not arrive as an exact string. 200 scenarios built from real OpenSanctions names, aliases and countries and real GLEIF names - exact hits, real aliases (LLC System Industrial Group is listed as LLC Sistema PG), typo near-matches, and coincidental namesakes in a different jurisdiction.
| 200 scenarios · real aliases + namesakes | Accuracy | Dangerous misses | False positives |
|---|---|---|---|
| Exact-name screenmembership in the sanctions list - the baseline | 60.0% | 80 | 0 |
| Fuzzy name-only screenrapidfuzz over names + aliases, no other signal | 62.5% | 0 | 40 |
| Composed: fuzzy × jurisdictionflag on the name, resolve identity by country | 99.5% | 0 | 0 |
Exact matching is dangerous 60% looks fine until you see the errors: 80 sanctioned-derived names waved through - every real alias and every typo variant. High apparent precision, zero recall on the cases that matter.
Fuzzy matching over-corrects It catches 100% of the real aliases but over-blocks 40 clean namesakes - it cannot tell a real hit from a coincidental name. In production that is alert fatigue and wrongful de-risking.
Composition resolves the trap Flag on the name, then resolve identity by jurisdiction: a fuzzy hit in a different country is a different entity. Zero dangerous misses and zero false positives.
Not 100% by construction The single error is a near-match classified No-Go instead of Abstain - over-strict, not dangerous. The composed screen can be wrong, which is the point: 99.5% is a real robustness number, honestly earned.
representation-bench-kyc/MATCHING.md.
// 200 scenarios · real OpenSanctions names + aliases + countries vs real GLEIF names · three screeners of increasing composition · dangerous misses (a sanctioned name screened Go) + false positives (a clean name blocked) · reproduce: CR-enterprise/benchmarks/representation-bench-kyc/matching_bench.py
The same architecture, a different domain: should we reorder this SKU? composed from camera (shelf reality), inventory (system on-hand), and POS (sales velocity). 204 scenarios over the real M5 (Walmart) data - and here we use the signals M5 actually ships: real price elasticity (measured per SKU from real sell_prices) and real event uplift (from the calendar), plus a camera that is allowed to lie. That is what makes the composed decision able to fail.
| 204 scenarios · real M5 calendar + prices | Accuracy | Stockouts missed | False reorders |
|---|---|---|---|
| Naive inventory-onlyreorder if system on-hand is low | 31.4% | 84 | 0 |
| Composed: trust camera + trailing velocitythe first benchmark's decider, on the harder set | 54.9% | 56 | 8 |
| Composed+: corroborate + calendar/pricereconcile camera × inventory × POS; real demand model | 96.1% | 0 | 8 |
The naive 100% was an artifact Add the signals M5 actually contains and a "trust the camera + trailing velocity" decider drops from 100% to 55%: it is velocity-blind (misses every promotion and event demand spike) and over-trusts a noisy camera. The earlier benchmark simply never tested these cases.
Composition recovers it, honestly Corroborating camera against POS kills the false reorders from a noisy camera (POS still selling ⇒ disbelieve the empty shelf, escalate), and the real calendar/price demand model catches the promo and event stockouts. Zero real stockouts missed.
The blind spot is reported A camera glitch that coincides with a real demand lull is observationally identical to a true stockout - no reconciliation over three signals can separate them. composed+ resolves toward restock (safe for real stockouts) and over-orders on all 8 traps. 96.1%, not hidden.
The phantom stockout is the retail analogue of KYC's "the sanction is on an owner upstream": inventory says stocked, the shelf is empty, and the answer is not in the representation you queried. Story Runtime is intentionally deferred - continuity metrics are not mature enough to score fairly yet.
retail-mission/DATASET.md + HARD.md.
// 204 scenarios · velocity, price elasticity and event uplift measured from real M5 Walmart series; on-hand + a fallible camera per the disclosed model · three deciders of increasing composition · reproduce: CR-enterprise/benchmarks/retail-mission/retail_hard.py
The four benchmarks measure each stage in isolation. End to end, one real onboarding question flows through all four in a single sub-10 ms run - each stage's output is the next stage's input, nothing mocked: the planner is real, the graph is real Neo4j.
// one workload, four planning stages, one trace · real planner (Context Runtime) + real Neo4j (GLEIF + OpenSanctions) · reproduce: CR-enterprise/benchmarks/stack-e2e/stack_trace.py