Better Context Beats More Context

Three lessons from stress-testing retrieval architectures against large models: where heavy infrastructure earned its cost, where it didn't, and why runtime orchestration still matters as models get better.
What we tested, and why the datasets matter
Every number below comes from a run on the same three large models, served locally for an apples-to-apples comparison. Retrieval and orchestration were driven by Context Runtime; a smaller model handled routing and graph/temporal extraction.
Contamination is the quiet threat in any RAG benchmark: if a model memorized the test set, "does context help" collapses. So we mix public datasets (transparent, reproducible) with a proprietary corpus the models could not have seen in training: the anti-parametric control.
| Role | Model | Notes |
|---|---|---|
| Answerer | Qwen3.6-Coder-Next | MoE, sparse active params |
| Answerer | NVIDIA Nemotron-3-Super | strong long-context handling |
| Answerer | DeepSeek-V4-Flash | heavy in-model context management |
| Router · extractor | Qwen3.6-27B | intent routing + graph/temporal extraction |
| Embeddings | BAAI/bge-large-en-v1.5 | dense retrieval |
| Judge | grok-4.5 | answer grading |
| Dataset | Availability | Regime it stresses |
|---|---|---|
| PopQA | public | single-fact lookup (low graph) |
| MuSiQue | public | multi-hop reasoning (graph) |
| LongMemEval | public | temporal / conversational memory |
| LiveRAG | public | mixed-domain, adversarial pollution |
| FinanceBench | public | numeric 10-K QA |
| Nutrition-consultation corpus | proprietary | anti-parametric (never in training) |
Lesson 1: Better architecture beats heavier infrastructure
One early assumption was wrong. We expected Graphiti's LLM-generated temporal graph to outperform simpler approaches. It didn't.
We paired each regime with its proper retriever (a real HippoRAG graph for multi-hop reasoning, our dependency-free SimGraph for temporal memory), then cross-applied the wrong one as a control, to rule out coincidence. The results ran in opposite directions.
On temporal data, Graphiti's LLM-extracted facts were lossy and lost to the simple SimGraph, which recovered perfect recall. On multi-hop graphs, the cheap 2-hop retriever collapsed: only HippoRAG's real entity graph could chain facts across passages.
Temporal recall — LongMemEval Multi-hop recall — MuSiQue
Graphiti (heavy) ███████████░░░░░ 0.73 SimGraph (light) ██████░░░░░░░░░░ 0.44
Document ████████████████ 1.00 Document █████████░░░░░░░ 0.65
SimGraph (light) ████████████████ 1.00 HippoRAG (heavy) ███████████████░ 0.98
↑ heavy engine LOSES ↑ heavy engine WINS
The same "does the expensive engine earn its cost?" question gets opposite answers by regime, and that asymmetry is the point.
| Regime | Simple (non-lossy) | Heavy (LLM engine) | Decision |
|---|---|---|---|
| Temporal | SimGraph 1.00 | Graphiti 0.73 | drop the heavy engine |
| Multi-hop graph | SimGraph 0.44 | HippoRAG 0.98 | keep the heavy engine |
The lesson was that heavy infrastructure should earn its cost: sometimes it does, sometimes it doesn't.
Lesson 2: Context quality matters more than context quantity
On clean data, DeepSeek-V4-Flash, Nemotron-3-Super and Qwen Coder-Next answered about equally well with the full context or Context Runtime's routed context, each with the retriever proper to the task.
The detail that matters: Context Runtime reached the same answer quality using roughly one-third of the context tokens. On multi-hop MuSiQue, routed context matched dumping every passage (a statistical tie), while beating naive retrieval by a wide, significant margin.
Context size (share of full-dump)
full-dump ████████████████████████ 100%
Context Runtime ████████ ≈33%
Answer accuracy vs full-dump: statistical tie (Δ ≈ +0.03, within noise)
Answer accuracy vs naive retrieval: +0.28 — decisive
It's about reaching the same answer using one-third of the context: lower latency, lower cost, more room for reasoning, and a larger safety margin before the window saturates. The effect held across an order of magnitude of model size, from 24–30B answerers to the 80–284B set.
Context quality beats context quantity. Better context beats bigger context.
Lesson 3: Runtime architecture still matters at scale
Frontier models increasingly perform sophisticated internal context management. We expected this to reduce the value of external context orchestration.
Instead, runtime orchestration still mattered once the context became large or polluted.
The evidence: DeepSeek-V4-Flash is exactly this kind of model: heavy internal context management. Yet under heavily polluted retrieval it began abstaining, returning "NOT FOUND" despite the answer being present. Context Runtime eliminated those failures by filtering and routing information before it ever reached the model.
DeepSeek-V4-Flash — "NOT FOUND" abstention rate (LiveRAG) clean │ polluted
full-dump ████████████████ 42% │ ██████ 17%
naive retrieval ██████ 17% │ ████████████████████ 50%
Context Runtime · 0% │ · 0% ← never drowns
Models with stronger long-context capabilities see smaller quality gains on clean data, but Context Runtime still cuts context usage substantially and keeps them robust once the workload becomes large or polluted. It's complementing modern models.
As models get better at reasoning, the competitive advantage shifts from model capability to context orchestration.
The through-line
This points at something almost nobody is saying yet: model progress and context orchestration are complementary. Better models increase the value of better context. They don't eliminate it.
Three findings, one shape. Heavy infrastructure should earn its cost. Context quality beats context quantity. A runtime layer should route signal to the model and avoid dumping noise at it.
Every generation of language models reasons better. None have infinite attention. Context Runtime's role is to make sure they spend their attention on the right information.
As models improve, context orchestration doesn't become less important. It becomes more efficient.
Methodology: models served locally (GPU NVFP4 + CPU GGUF), Context Runtime for routing/retrieval, grok-4.5 as judge. Retrieval metrics are answer-model-independent; the DeepSeek abstention result is judge-free. Public datasets are openly available; the nutrition corpus is proprietary and used specifically so no model could have seen it in training.
Data sources and licensing: PopQA (MIT), MuSiQue (CC-BY-4.0), and LongMemEval (MIT), each used under its own license; loaders download them on demand and we do not redistribute them. The nutrition corpus is a proprietary internal set built from third-party lecture content with no open license - reported as aggregate recall only, not redistributed, and used purely as an anti-contamination control.
#ContextEngineering #RAG #LLM #KnowledgeGraphs #HippoRAG #Graphiti #AIEngineering #LLMOps #RetrievalAugmentedGeneration #ContextRuntime #DeepSeek #Nemotron #Qwen #AIInfrastructure #MLOps #GenAI #ContextOrchestration #Benchmarks