All articles
    rag
    knowledge-graphs
    context-engineering
    hipporag
    graphiti
    benchmarks
    llm-optimization

    Why Context Runtime Started Routing Knowledge Instead of Retrieving It

    July 11, 2026redevops.io
    Why Context Runtime Started Routing Knowledge Instead of Retrieving It

    Most RAG systems assume there is a single best knowledge representation. Our benchmarks challenged that assumption.

    We tested three real retrieval engines, document (BM25/hybrid), graph (HippoRAG), and temporal (Graphiti), across three reasoning regimes. The runtime's first task was deciding which knowledge representation should answer it.

    The short version:

    • An LLM router sent queries to the right representation 100% / 90% / 100% of the time. The keyword router managed 95% / 0% / 0%, blind to implicit intent.
    • Real HippoRAG lifted multi-hop retrieval recall 0.65 → 0.99.
    • Routed context matched "dump everything" on judged answers at ~⅓ the tokens, and clearly beat naive retrieval.
    • On a tiny temporal haystack, the graph-temporal engine lost to plain retrieval. We publish that too.

    The core idea: knowledge representation is a planning decision

    The industry reflex to a hard retrieval problem is a bigger model or a bigger context window: both cost money and latency. Context Runtime v4 takes a different route, classify → constrain → learn: classify the query into a knowledge representation, constrain retrieval to that engine, then let an online bandit learn the best method per context.

                  ┌─────────────┐
                  │ User query  │
                  └──────┬──────┘
                         ▼
                ┌────────────────┐
                │  Classify the  │
                │ representation │
                └──┬─────┬─────┬─┘
            single │     │     │ history
             fact  ▼     ▼multi-hop
          ┌─────────┐ ┌────────┐ ┌───────────┐
          │Document │ │ Graph  │ │ Temporal  │
          │BM25/hyb │ │HippoRAG│ │ Graphiti  │
          └────┬────┘ └───┬────┘ └─────┬─────┘
               └──────────┼────────────┘
                          ▼
                 ┌─────────────────┐
                 │  Bandit learns  │
                 │ best method/ctx │
                 └────────┬────────┘
                          ▼
                   ┌─────────────┐
                   │ Answer model│
                   └─────────────┘
    

    To test it honestly, the "right" answer for the router had to come from the dataset's design, independent of the routing model.

    Public datasets: one per regime

    RegimeDatasetSource (public)Why it's here
    Low / no graphPopQAakariasai/PopQASingle-fact lookups: document should win
    Multi-hop graphMuSiQuedgslibisey/MuSiQue2-hop over 20 paragraphs: graph's turf
    Temporal conversationLongMemEval (oracle)xiaowu0162/longmemevalTimestamped chat sessions: bi-temporal's turf

    Exact models: all local, all NVFP4, one 48 GB GPU

    RoleModelLicense
    Router · extractor · answererQwen3.6-27B-NVFP4 (prithivMLmods/Qwen3.6-27B-NVFP4)Apache-2.0
    AnswererMistral-Small-3.2-24B-Instruct-2506-NVFP4Apache-2.0
    AnswererGemma 26B-A4B (NVFP4)Apache-2.0
    AnswererNVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4NVIDIA Open Model License
    EmbeddingsBAAI/bge-large-en-v1.5 (1024-dim, CPU)MIT
    Judgegrok-4.5 (xAI), 578 rows judged for $0.51n/a

    Every answerer is 24–30B, deliberately no bigger model in the set, so this is a clean test of better context vs. more context at equal model size.

    Real engines, real repos

    LayerRepository
    Orchestrationgithub.com/redevops-io/context-runtime · github.com/redevops-io/redevops-rag
    Graph enginegithub.com/OSU-NLP-Group/HippoRAG: OpenIE graph + Personalized PageRank
    Temporal enginegithub.com/getzep/graphiti: Neo4j bi-temporal knowledge graph

    Results

    There was no universal winner, which is exactly the point.

    1) Intelligent routing is the whole ballgame

    Routing accuracy (→ correct representation)         keyword ░   LLM █
    
    PopQA   → document   keyword ███████████████████░ 0.95    LLM ████████████████████ 1.00
    MuSiQue → graph      keyword ░░░░░░░░░░░░░░░░░░░░ 0.00    LLM ██████████████████░░ 0.90
    LongMem → temporal   keyword ░░░░░░░░░░░░░░░░░░░░ 0.00    LLM ████████████████████ 1.00
    

    The shipped keyword router collapses to 0.00 on the two hard regimes: "Who is the spouse of the Green performer?" has no "multi-hop" keyword, so it defaults to document and the graph engine is never reached. The LLM router nails all three.

    The strongest test is one shuffled stream, so the router must decide per query from the query alone: low-graph 100% → document, multi-hop 93% → graph, temporal 100% → temporal.

    2) On multi-hop, the graph engine earns its keep

    MuSiQue retrieval recall (bootstrap 95% CIs)
    
    fixed graph (HippoRAG)   ███████████████████▓ 0.988  [0.963, 1.000]
    CR (routed + learned)    ██████████████████▌  0.925  [0.863, 0.988]
    fixed document           █████████████        0.650  [0.550, 0.738]
    

    Δ(graph − document) = +0.338, 95% CI [0.250, 0.438], robustly significant. And you can watch the bandit learn it: cumulative recall climbed 0.750 → 0.850 → 0.900 → 0.925 as the chosen arm shifted toward graph.

    3) Better context ≈ more context: at a third of the cost

    Judged answer accuracy (grok-4.5), averaged across all four models:

    MuSiQue answer accuracy
    
    CR-routed context   ████████████████████ 0.49   ← ~⅓ the tokens of full-dump
    full-dump (all 20)  ██████████████████▌  0.46
    naive (single-hop)  ███████████▌         0.29
    
    • CR vs full-dump: Δ = +0.037, 95% CI [−0.025, +0.106] → a statistical tie. Matching "dump everything" while using ~⅓ the tokens is the win.
    • CR vs naive: Δ = +0.206, 95% CI [+0.144, +0.269] → CR significantly wins.
    • CR ≥ full-dump on every individual model.

    PopQA ties across all methods (~0.97), precisely why routing to the cheapest representation there is an efficiency win.

    4) The honest negative: temporal on a small haystack

    We don't cherry-pick. On LongMemEval, routing was perfect (100% → temporal), but the temporal engine lost: Graphiti session-recall 0.731 vs document 1.000; judged answers full-dump 0.23 ≫ CR 0.03.

    Why: the oracle split keeps only ~3 evidence sessions per question, a haystack so small that hybrid search finds every relevant one, and Graphiti's LLM-extracted facts are lossy versus the raw turns. Graphiti's real advantage (point-in-time reasoning over a large, revised history) needs the full haystack (hundreds of distractor sessions) to show. On this dataset, the correct representation is document. (Robustness check: doubling the efficiency penalty didn't change PopQA's route.)

    The engineering takeaway: ship the LLM intent head

    The benchmark's clearest signal: the v4 architecture is right, but its shipped keyword head scores 0.00 on implicit intent. The fix is a hybrid, confidence-gated analyzer, keyword-first (free on explicit cues), LLM only when the cheap head is unsure:

       Query
         │
         ▼
     ┌──────────────────┐   confident non-document rep
     │ Keyword analyzer │ ───────────────────────────►  use it (free)
     │      (free)      │
     └────────┬─────────┘
              │ document default / low confidence
              ▼
       ┌──────────────┐   agrees / unreachable
       │   LLM head   │ ──────────────────────────►  keep keyword verdict
       └──────┬───────┘
              │ override
              ▼
       new representation  +  re-align model tier
    

    Implicit multi-hop → 1 LLM call → graph; explicit temporal cue → 0 LLM calls; LLM down → graceful keyword fallback. It carries a cost-ledger so the per-query routing overhead is measurable: the number you weigh against a model's built-in context handling.

    Why it matters

    This validated a core idea behind Context Runtime v4: knowledge representation is a planning decision, the choice of retrieval strategy. That now extends into Mission Runtime, which plans over knowledge representations, capabilities, models, applications, humans, and execution strategies, so enterprise AI will be differentiated by runtime architecture.

    So before reaching for a bigger model or context window, ask whether you're even retrieving the right kind of knowledge, and let the honest failure mark exactly where the frontier still is.


    Reproducibility: reward = recall − λ·cost (λ=0.15), seed=13, bandit ε=0.15; all NVFP4 on one 48 GB GPU. Engines: context-runtime, redevops-rag, OSU-NLP-Group/HippoRAG, getzep/graphiti. Datasets: akariasai/PopQA, dgslibisey/MuSiQue, xiaowu0162/longmemeval.

    Data sources and licensing: the benchmarks use public datasets under their own licenses - PopQA (MIT), MuSiQue (CC-BY-4.0), and LongMemEval (MIT). Loaders download the data on demand; we do not redistribute it.

    Built on open source. Every number is real, including the one that didn't go our way.

    #ContextEngineering #RAG #KnowledgeGraphs #LLM #LLMOps #AIEngineering #MachineLearning #HippoRAG #Graphiti #Neo4j #RetrievalAugmentedGeneration #OpenSource #NVFP4 #GenAI #MLOps #AIInfrastructure

    Want ReDevOps in your Google Top Stories & AI results?Follow ReDevOps on Google