← redevops.io  /  Benchmarks  /  Representation Retrieval ↓   Reasoning ↔
Representation planning · the decision above retrieval

Which knowledge representation?

Before you can pick a retriever or a reasoning strategy, one decision comes first: which representation of knowledge should answer this question at all. Get it wrong and no retriever tuning saves you - searching a document index for a five-hop ownership chain is not a bad query, it is the wrong access path. This is Stage 3 of the planning stack, and the benchmark that did not exist yet.

Evidence · synthetic · internally reproduced 27 Oct 2026: 278/300 correct (95% interval 89.7 to 95.7), the same decisions on context-runtime v0.1.0, v0.2.0, v0.3.0 and the current development and production lines; the best fixed representation scores 50/300 (chance 16.7%). Selected-engine routing on the 59 labelled KYC queries reproduced as well: 54/59 and 56/59, identical on every version.

the planning stack:   representation → retrieval → reasoning → mission  |  you are here: representation

The decision the planner makes, by example - the same intent, routed to the representation that can answer it:

QuestionCorrect representation
"revenue by region last quarter"analytical · SQL
"who ultimately owns this business"graph · property graph
"the deployment state as of the incident"temporal · state_at(T)
"what the runbook says about rollbacks"document · BM25 / dense
"where the validate_token function is defined"code · code-aware
"what the shelf photo shows"multimodal · vision

RepresentationBench

300 curated enterprise questions, 50 per representation (analytical, document, graph, temporal, code, multimodal), each run through the real planner - intent analysis → candidate generation → cost-model selection. We read back the chosen representation, the concrete method, the confidence, and the decision latency. Two decisions are measured: which representation, and which method inside it.

Representation (50 questions each)Accuracy
analytical (SQL)"how many X per Y" · "top 10 by" · "breakdown of"
100%
document"what does the runbook say" · "find the postmortem"
100%
graph"who ultimately owns X" · "what depends on X" · "blast radius"
90%
temporal"state of X as of 2021" · "what changed since the release"
86%
code"where the validate_token function is" · "the patch that fixed the bug"
90%
multimodal"what the shelf photo shows" · "extract the total from the invoice image"
90%
92.7% representation-choice 92.8% method routing 0.12 ms decision · p50 278 / 300 correct · every decision sub-millisecond

Representation planning picks the representation; the cost model then picks the method inside it - the same graph question can be served by a cypher traversal or a generic hybrid search, and the planner must prefer the specialist. 128 / 138 = 92.8% of correctly-represented analytical/graph/temporal questions also get the specialist method (sql / cypher / temporal), not the fallback. code and multimodal are recognised as representations but have no specialist retriever bound yet - both fall back to hybrid retrieval; reported as an implementation-status gap, not a routing miss.

The choice is legible Every decision carries its explanation - the intent bucket, the chosen representation and method, and a confidence - so routing reads as "aggregate cue → analytical → SQL," auditable, not a black box.

Sub-millisecond The planner decision is p50 0.11 ms, p95 0.23 ms - representation selection is not a model call, it is a classify-and-score over an inspectable candidate set.

Honest misroutes (22/300) "which services call X" (call is too common a word to be a graph trigger), "the version before the patch" (bare before is too broad a temporal cue). One is a genuine composition case: "extract the total from the invoice image" fires the analytical hint (total) before the multimodal one - read-the-image-and-aggregate, which a single-label benchmark scores as a miss. Each names a bounded lever; none is a collapse.

The benchmark drove a fix The ownership hint matched "owns" but not the plural "owners" - a word-boundary defect that dropped "beneficial owners" to document. Fixing it lifted graph routing and the KYC routing 93.2% → 94.9%. Building the benchmark improved the planner.

// 300 curated questions · 50/representation × 6 · run through the real planner (intent → candidates → cost-model select) · representation-choice + method-routing accuracy + decision latency + explanation · reproduce: CR-enterprise/benchmarks/representation-bench/

Above this: mission planning

Representation planning chooses one access path. The stage above composes several into a verified decision. It is live: RepresentationBench-KYC scores the planner's compose-verify-abstain on a real Go/No-Go onboarding decision - 100% vs a naive single-node screen at 36.7%, which onboarded 45 of 50 sanctioned-linked entities because the sanction sits on an owner upstream. A harder matching-robustness half (real aliases + namesakes) scores 99.5% where a name-only screen is a precision/recall trap, and a Retail restock-or-abstain benchmark over real Walmart demand rounds out the stage. The architecture is in the v7 whitepaper.

The planning stack, measured: representation → retrieval → reasoning → mission. The layer behind it: v7 - the AI query planner.
Data sources & licensing
RepresentationBench's 300 questions are authored - deterministic templates over fictional entities (no external dataset, no query log), so there is no third-party data to license here. The mission and retrieval stages it links to do use public data; their sources and licenses (GLEIF CC0, OpenSanctions CC-BY-NC non-commercial, M5, PopQA, MuSiQue, LongMemEval, TempReason) are declared on the mission and retrieval pages.
redevops.io is a member of the NVIDIA Inception program. © 2026 NVIDIA, the NVIDIA logo and NVIDIA Inception are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries.