Bigger Context Windows Won't Save You Money.

A larger context window lets you send more data. It does not tell you whether that data belongs there.
Every unnecessary token costs:
- latency
- memory bandwidth
- GPU utilization
- verification time
- model attention
- money
The expensive part is the model call and the decision to send irrelevant context in the first place.
The bad default
Most AI apps still use static rules:
- always retrieve top-10
- always rerank
- always attach memory
- always expose all tools
- always verify with a stronger model
- always include the full schema
That works in demos. It becomes expensive in production.
Some requests need five documents. Some need one row. Some need no retrieval. Some need a local model, some a frontier model, some verification. Most do not need all of it.
The hidden cost of MCP/tool sprawl
Adding tools feels free. It is not.
If every MCP server, function schema, database definition, skill, and retriever is injected into the prompt, capability becomes context.
More capabilities then mean:
- larger prompts
- more irrelevant definitions
- more tool confusion
- more latency
- more cost
That is the wrong scaling model.
Capabilities should live in a registry. The planner should select them. The model should only see the chosen surface.
The remedy
Context should be planned like a query.
Before the first model call, the runtime should decide:
- retrieve or skip retrieval
- sparse, dense, hybrid, graph, memory, or domain route
- local model or frontier model
- verify or abstain
- which plugins are allowed
- which context is worth paying for
Then measure the result and improve the next plan.
This is the difference between:
bigger context
and
better context
The point
The winner will be the system that knows when not to use the largest context window.
Reference implementation: https://github.com/redevops-io/context-runtime
Benchmarks: https://redevops.io/benchmarks
Whitepaper v2: https://redevops.io/whitepaper-v2
redevops.io builds rAIlsApps (source-available (AGPL-3.0 with the Commons Clause)): self-hostable modules run by agents, on hardware you can actually afford. See it run.

