← redevops.io
Private pitch · redevops.io

Context Runtime

This paper documents the capabilities introduced in this version. Later versions extend rather than replace this architecture — see Whitepaper v4 for current status.
The Evolution of AI Execution Planning
Where this fits. v1–v3 are the Context Runtime foundation — intent over hardcoded pipelines, candidate generation, cost/quality/latency planning, retrieval and model routing, verification and learning. The current system-level reference is Whitepaper v8; the canonical model is the Platform Architecture spec — where an older description here disagrees with the spec, the spec wins.
Applications describe intent. The runtime selects the execution strategy, explains it, and keeps it correct as the world drifts — under policy.

AI execution planning is a moving target

Traditional database query planners assume a mostly stationary world. Statistics converge over time, letting a planner estimate the cost of alternative execution strategies without running them.

AI systems operate under fundamentally different conditions. Models evolve. Knowledge bases change. Providers modify pricing and latency. User behavior shifts. Retrieval quality drifts. Verification policies change. Unlike databases, AI execution planning is a continuously moving optimization problem.

How Context Runtime handles a request
Figure 1 — How a request is handled. Capabilities are plugins the planner selects from (a registry, out of band), not MCP tools injected into the model's context — so the context stays clean no matter how many data sources, tools, or skills you add.

The five generations

We believe AI infrastructure will evolve through distinct generations, each addressing a limitation of the previous one.

Generation 1 — Static cost models

Execution plans are selected using predefined heuristics: fixed model routing, fixed Top-K retrieval, static reranking, rule-based verification. Most current AI applications operate here.

Generation 2 — Measured cost models

Execution decisions are informed by observed production statistics — measured latency, token consumption, retrieval quality, verification success, and user feedback. Execution statistics become empirical rather than heuristic. Context Runtime records these measurements and uses them to recalibrate estimates today. Generation 4 adds controlled exploration, uncertainty management, and adaptation under drift, preventing the learner from becoming trapped by the plans it already prefers.

Generation 3 — Observable execution planning

Optimization alone is insufficient — developers must understand why a strategy was selected. For each request the runtime exposes candidate plans, rejected alternatives, estimated vs actual cost, retrieval evidence, verification decisions, and reward provenance. Trust requires observability. (Shipped as EXPLAIN.)

Generation 4 — Online learning execution planning

Even observable planners go stale: rejected plans stop producing observations, providers improve, knowledge bases drift. The planner must occasionally explore alternative strategies to keep its estimates accurate. Context Runtime extends the measured cost model into an online framework using contextual bandits and controlled exploration — balancing exploration against exploitation so planning becomes a continuously adapting service rather than a static optimization problem.

Generation 5 — Trust-aware execution

Production AI systems ultimately fail for one reason: humans stop trusting them. Research across security, BI, CRM, coding agents, support, and enterprise RAG revealed the same pattern — confident-but-wrong recommendations, opaque decisions, and noisy automation eventually cause users to revert to manual workflows. The failure is not simply hallucination; it is trust collapse.

Future planners should optimize not only latency, cost, and quality, but also trust — a measurable objective derived from acceptance rates, manual overrides, regeneration requests, verification outcomes, calibrated confidence, and abstentions. The runtime learns not only which strategy is cheapest, but which one operators are willing to rely on.

Improved execution planning, v1 to v2
Figure 2 — v1 → v2. v1 picks one plan by a static rule; v2 scores candidate plans on measured cost and quality (Gen 2), explains the choice (Gen 3, EXPLAIN), and learns from served outcomes; Generation 4 adds explicit exploration and adaptation under drift.

Online optimization — remaining correct in a non-stationary world

The generations raise a sharper question than "which plan is cheapest": how does a planner remain correct in a non-stationary world? A database query optimizer may assume its statistics converge; an AI execution planner cannot. Models improve, knowledge bases evolve, providers change pricing and latency, verification policies shift, and user expectations move. AI execution planning cannot assume convergence.

Context Runtime v3 — online optimization under drift
Figure 3 — v3 keeps the planner correct as the world drifts, spending the cheapest form of exploration that can answer the question: off-policy evaluation from the logs, then shadow execution off the serving path, then live exploration only for genuinely novel contexts.

The exploration problem

Measured execution alone is not enough. As the planner increasingly favors the strategies that have worked, the alternatives become under-sampled, and it gradually loses information about the plans it no longer runs. Left unchecked, a "measured" planner slowly goes blind to everything outside its current habit. This is an exploration problem, not merely an optimization problem.

Exploration under drift — a layered strategy

A production planner must keep learning without degrading the experience of the users it is serving, spending the cheapest form of exploration that can answer the question at hand:

Because production systems drift continuously, historical observations gradually lose predictive value. Context Runtime is therefore designed to rebalance recent against historical evidence rather than converge to a fixed policy: discounted updates that fade stale evidence, windowed estimation, uncertainty that widens when the world changes, and detection of abrupt shifts. We state this as a property, not an algorithm:

The runtime must maintain sufficient uncertainty about alternative execution strategies to remain adaptive under environmental drift.

This is where the database analogy intentionally diverges. A query optimizer optimizes over relatively stable statistics; Context Runtime optimizes over continuously changing outcomes. The objective is not to discover a single optimal plan and keep it — it is to continuously maintain the best available execution policy.

                 Goal
                  │
                  ▼
           Candidate plans
                  │
                  ▼
           Cost model            (Gen. 1 — static estimates)
                  │
                  ▼
        Measured execution       (Gen. 2 — empirical recalibration)
                  │
                  ▼
      Explain / observability    (Gen. 3 — every decision inspectable)
                  │
                  ▼
        Online optimization      (Gen. 4 — exploration under drift)
                  │
                  ▼
      Adaptive execution policy  (maintained, never frozen)

“Gen. N” is the architectural generation, not the whitepaper version — the two series only partly align. This paper (Whitepaper v3) introduces Generations 3–5; knowledge-aware planning is Generation 6, in Whitepaper v4.

Policy-constrained planning (enterprise governance)

One thing missing from nearly every agent framework is governance. Production AI requires more than retrieval optimization — the runtime must also determine what information an execution is allowed to access, and under what constraints. Context Runtime treats governance as part of planning rather than enforcement bolted on afterward.

RBAC is too narrow. This is policy-aware planning: the planner optimizes under constraints. Policies define the feasible execution space before the optimizer ranks candidates. A plan that violates policy is infeasible regardless of its estimated quality or cost — authorization becomes a first-class planning constraint.

Goal → Policy Engine → Feasible Plans → Cost Model → Planner → Execution

Constraints are far richer than "can access table X":

Policies apply at many levels: application · tenant · user · agent · tool · knowledge source · database · schema · table · document · collection · row · column · file · object-storage path. The commercial value of the enterprise edition is a planner that can prove every execution stayed within security, compliance, cost, and governance constraints while still optimizing for quality — complementing the open-source optimization engine rather than competing with it.

The assistant pattern — one shared agent, not one per session

A natural instinct when adding an assistant to every application is to imagine one assistant instance per application, per user, per session — which implies unbounded token consumption. That framing mistakes the unit of scale. An assistant is not a persistent process; it is a sequence of stateless requests. A session is persisted state, not a continuously running model process. Nothing computes between turns.

Decompose the assistant by required capability, and most of it is not frontier-model work:

The token-cost fear dissolves because compute is routed per request, not reserved per instance. One shared assistant capability serves the whole fleet — the same code, the same model pool — parameterized per application by identity, scope, and task.

Scaling Context Runtime in the enterprise

Stateless planning, horizontal replication

Planning a request is stateless: intent classification, feasibility filtering, cost ranking, and executor selection are pure functions of the request plus the current policy snapshot. Stateless planners replicate trivially — one instance per pod, behind a load balancer, scaled on queries-per-second. The only shared state is the learned policy (cost-model statistics, the bandit's per-context values, calibration maps, the trust ledger), externalized to a shared store so every replica plans against the same learned world.

The learning loop is asynchronous

Heavy aggregation and model updates should remain off the request's hot path; lightweight outcome recording may occur synchronously. Each execution emits an outcome event — the EXPLAIN trace, actual cost and latency, verification result, and the human signal (acceptance, override, regeneration). Those events flow to an aggregator that folds them into the cost model, bandit, calibration, and trust ledger, and publishes a new policy snapshot replicas pick up. Load grows without adding latency, because getting smarter is decoupled from answering. A log/event backbone (Apache Kafka is the natural fit) carries both halves — context-freshness updates to the indexer, and outcome traces to the policy aggregator. Neither stream ever touches the model's context window.

Many data sources, and OLAP at scale

Enterprises have OLTP systems, a warehouse, object storage, vector stores, and increasingly column-oriented OLAP engines — ClickHouse, Apache Doris. Context Runtime's design treats each as an adapter behind a uniform plugin interface — a PostgresAdapter, a ClickHouseAdapter, a DorisAdapter, a vector retriever — all exposing the same search / query contract, so the cost model can learn each engine's real cost and route a question to the cheapest engine that can answer it. Multiple data sources become a cost-based engine-selection problem, and governance is expressed as data-scope constraints in the policy layer. (Status: the analytical adapters are the roadmap — a Doris binding exists in the enterprise edition; the document, graph, temporal, and community representations ship today.)

The pivotal property: capabilities live in the planner, not the prompt

If we let users query ClickHouse or Doris instead of Postgres, do we have to write new agent "skills," stand up MCP servers, and feed their schemas to the model — bloating the context? No. Data sources, query templates, tools, and representations are registered out of band; the planner — not the prompt — selects among them. Swapping Postgres for ClickHouse or Doris is an adapter-and-template change, invisible to the model's context. For the rare generative path, the runtime supplies schema on demand — the minimal schema of the specific tables the plan touches, scoped by policy — never the whole catalog.

Capability count scales in a registry — which is cheap — rather than in the context window, which is not.

The next evolution

GenerationStageStatus
1Static cost modelsIndustry default
2Measured cost modelsContext Runtime today
3Observable execution planningShipped (EXPLAIN)
4Online learning execution planningBandit + exploration (partial)
5Trust-aware executionRoadmap
—Policy-constrained planningEnterprise open-core

Generation 5 combines trust-aware optimization with policy-constrained planning: trust is the objective; policy defines the feasible execution space.

Bridge to knowledge-aware planning

Online optimization can only select among capabilities exposed to it. Historical conversational workloads revealed questions that document, hybrid and multi-hop retrieval cannot answer reliably: what was true at a specific time, and how was it later superseded? Whitepaper v4 introduces temporal knowledge as a first-class representation and generalizes this observation into knowledge-aware planning.

Context Runtime — private pitch. Not for distribution. © 2026 redevops.io  ·  Whitepaper v3 — The Evolution of AI Execution Planning