Traditional database query planners assume a mostly stationary world. Statistics converge over time, letting a planner estimate the cost of alternative execution strategies without running them.
AI systems operate under fundamentally different conditions. Models evolve. Knowledge bases change. Providers modify pricing and latency. User behavior shifts. Retrieval quality drifts. Verification policies change. Unlike databases, AI execution planning is a continuously moving optimization problem.
We believe AI infrastructure will evolve through distinct generations, each addressing a limitation of the previous one.
Execution plans are selected using predefined heuristics: fixed model routing, fixed Top-K retrieval, static reranking, rule-based verification. Most current AI applications operate here.
Execution decisions are informed by observed production statistics — measured latency, token consumption, retrieval quality, verification success, and user feedback. Execution statistics become empirical rather than heuristic. Context Runtime records these measurements and uses them to recalibrate estimates today. Generation 4 adds controlled exploration, uncertainty management, and adaptation under drift, preventing the learner from becoming trapped by the plans it already prefers.
Optimization alone is insufficient — developers must understand why a strategy was selected. For each request the runtime exposes candidate plans, rejected alternatives, estimated vs actual cost, retrieval evidence, verification decisions, and reward provenance. Trust requires observability. (Shipped as EXPLAIN.)
Even observable planners go stale: rejected plans stop producing observations, providers improve, knowledge bases drift. The planner must occasionally explore alternative strategies to keep its estimates accurate. Context Runtime extends the measured cost model into an online framework using contextual bandits and controlled exploration — balancing exploration against exploitation so planning becomes a continuously adapting service rather than a static optimization problem.
Production AI systems ultimately fail for one reason: humans stop trusting them. Research across security, BI, CRM, coding agents, support, and enterprise RAG revealed the same pattern — confident-but-wrong recommendations, opaque decisions, and noisy automation eventually cause users to revert to manual workflows. The failure is not simply hallucination; it is trust collapse.
Future planners should optimize not only latency, cost, and quality, but also trust — a measurable objective derived from acceptance rates, manual overrides, regeneration requests, verification outcomes, calibrated confidence, and abstentions. The runtime learns not only which strategy is cheapest, but which one operators are willing to rely on.
The generations raise a sharper question than "which plan is cheapest": how does a planner remain correct in a non-stationary world? A database query optimizer may assume its statistics converge; an AI execution planner cannot. Models improve, knowledge bases evolve, providers change pricing and latency, verification policies shift, and user expectations move. AI execution planning cannot assume convergence.
Measured execution alone is not enough. As the planner increasingly favors the strategies that have worked, the alternatives become under-sampled, and it gradually loses information about the plans it no longer runs. Left unchecked, a "measured" planner slowly goes blind to everything outside its current habit. This is an exploration problem, not merely an optimization problem.
A production planner must keep learning without degrading the experience of the users it is serving, spending the cheapest form of exploration that can answer the question at hand:
Because production systems drift continuously, historical observations gradually lose predictive value. Context Runtime is therefore designed to rebalance recent against historical evidence rather than converge to a fixed policy: discounted updates that fade stale evidence, windowed estimation, uncertainty that widens when the world changes, and detection of abrupt shifts. We state this as a property, not an algorithm:
This is where the database analogy intentionally diverges. A query optimizer optimizes over relatively stable statistics; Context Runtime optimizes over continuously changing outcomes. The objective is not to discover a single optimal plan and keep it — it is to continuously maintain the best available execution policy.
Goal
│
▼
Candidate plans
│
▼
Cost model (Gen. 1 — static estimates)
│
▼
Measured execution (Gen. 2 — empirical recalibration)
│
▼
Explain / observability (Gen. 3 — every decision inspectable)
│
▼
Online optimization (Gen. 4 — exploration under drift)
│
▼
Adaptive execution policy (maintained, never frozen)“Gen. N” is the architectural generation, not the whitepaper version — the two series only partly align. This paper (Whitepaper v3) introduces Generations 3–5; knowledge-aware planning is Generation 6, in Whitepaper v4.
One thing missing from nearly every agent framework is governance. Production AI requires more than retrieval optimization — the runtime must also determine what information an execution is allowed to access, and under what constraints. Context Runtime treats governance as part of planning rather than enforcement bolted on afterward.
RBAC is too narrow. This is policy-aware planning: the planner optimizes under constraints. Policies define the feasible execution space before the optimizer ranks candidates. A plan that violates policy is infeasible regardless of its estimated quality or cost — authorization becomes a first-class planning constraint.
Goal → Policy Engine → Feasible Plans → Cost Model → Planner → Execution
Constraints are far richer than "can access table X":
Policies apply at many levels: application · tenant · user · agent · tool · knowledge source · database · schema · table · document · collection · row · column · file · object-storage path. The commercial value of the enterprise edition is a planner that can prove every execution stayed within security, compliance, cost, and governance constraints while still optimizing for quality — complementing the open-source optimization engine rather than competing with it.
A natural instinct when adding an assistant to every application is to imagine one assistant instance per application, per user, per session — which implies unbounded token consumption. That framing mistakes the unit of scale. An assistant is not a persistent process; it is a sequence of stateless requests. A session is persisted state, not a continuously running model process. Nothing computes between turns.
Decompose the assistant by required capability, and most of it is not frontier-model work:
The token-cost fear dissolves because compute is routed per request, not reserved per instance. One shared assistant capability serves the whole fleet — the same code, the same model pool — parameterized per application by identity, scope, and task.
Planning a request is stateless: intent classification, feasibility filtering, cost ranking, and executor selection are pure functions of the request plus the current policy snapshot. Stateless planners replicate trivially — one instance per pod, behind a load balancer, scaled on queries-per-second. The only shared state is the learned policy (cost-model statistics, the bandit's per-context values, calibration maps, the trust ledger), externalized to a shared store so every replica plans against the same learned world.
Heavy aggregation and model updates should remain off the request's hot path; lightweight outcome recording may occur synchronously. Each execution emits an outcome event — the EXPLAIN trace, actual cost and latency, verification result, and the human signal (acceptance, override, regeneration). Those events flow to an aggregator that folds them into the cost model, bandit, calibration, and trust ledger, and publishes a new policy snapshot replicas pick up. Load grows without adding latency, because getting smarter is decoupled from answering. A log/event backbone (Apache Kafka is the natural fit) carries both halves — context-freshness updates to the indexer, and outcome traces to the policy aggregator. Neither stream ever touches the model's context window.
Enterprises have OLTP systems, a warehouse, object storage, vector stores, and increasingly column-oriented OLAP engines — ClickHouse, Apache Doris. Context Runtime's design treats each as an adapter behind a uniform plugin interface — a PostgresAdapter, a ClickHouseAdapter, a DorisAdapter, a vector retriever — all exposing the same search / query contract, so the cost model can learn each engine's real cost and route a question to the cheapest engine that can answer it. Multiple data sources become a cost-based engine-selection problem, and governance is expressed as data-scope constraints in the policy layer. (Status: the analytical adapters are the roadmap — a Doris binding exists in the enterprise edition; the document, graph, temporal, and community representations ship today.)
If we let users query ClickHouse or Doris instead of Postgres, do we have to write new agent "skills," stand up MCP servers, and feed their schemas to the model — bloating the context? No. Data sources, query templates, tools, and representations are registered out of band; the planner — not the prompt — selects among them. Swapping Postgres for ClickHouse or Doris is an adapter-and-template change, invisible to the model's context. For the rare generative path, the runtime supplies schema on demand — the minimal schema of the specific tables the plan touches, scoped by policy — never the whole catalog.
| Generation | Stage | Status |
|---|---|---|
| 1 | Static cost models | Industry default |
| 2 | Measured cost models | Context Runtime today |
| 3 | Observable execution planning | Shipped (EXPLAIN) |
| 4 | Online learning execution planning | Bandit + exploration (partial) |
| 5 | Trust-aware execution | Roadmap |
| — | Policy-constrained planning | Enterprise open-core |
Generation 5 combines trust-aware optimization with policy-constrained planning: trust is the objective; policy defines the feasible execution space.
Online optimization can only select among capabilities exposed to it. Historical conversational workloads revealed questions that document, hybrid and multi-hop retrieval cannot answer reliably: what was true at a specific time, and how was it later superseded? Whitepaper v4 introduces temporal knowledge as a first-class representation and generalizes this observation into knowledge-aware planning.