← redevops.io
Draft · v10 · redevops.io

Reference-First Context

A governed harness for model execution — how models interact with large application state without serializing it into the prompt
v10 is the harness paper. v7 chose representations and access paths; v8 decided when work should happen and learned which strategy works; v9 executed the runtime operators portably. v10 answers the remaining question: once the planner has selected artifacts and capabilities, how should a model safely and efficiently interact with them without copying the world into its context?

Thesis. Models should not receive application state as serialized prompt content. They should operate over versioned references to governed artifacts, expanding only the context the current decision requires. Put in database terms: Context Runtime treats artifacts as database objects, prompts as bounded execution views, and model context as a planned working set — not a serialized copy of application state.

Architectural Law — the boundary the whole paper rests on:
  Models never observe artifacts directly. They observe only ContextViews materialized by the runtime.

  Artifact
     │   governed reference
  ArtifactHandle
     │   planner: what is required?
  ContextPreviewPlan
     │   harness: context materialization
  ContextView            ← a first-class, content-addressed runtime artifact
     │
  Model
     │   typed, authorized, logged, replayable
  RuntimeEvents  (DereferenceEvent, …)

1. The serialization ceiling

The default pattern assembles a prompt from everything in scope — application state, retrieved documents, prior messages, worker outputs, tool results — and serializes it. That fails on context limits, repeated tokens, cache invalidation, stale copies, lost provenance, authority ambiguity, accidental disclosure, inconsistent summaries, and an inability to reason over repositories or evidence graphs larger than one call. Larger context windows and provider cache reuse delay this failure; they do not fix the architecture. The differentiator is governance — authority, provenance, staleness, and disclosure are only enforceable when context is constructed by reference, not by copy.

2. Artifacts, handles, views — and materialization

Three distinct concepts. An Artifact is an immutable or versioned source object. An ArtifactHandle is a stable typed reference carrying enough metadata to plan without exposing the object: id, type, version/content hash, authority, visibility, owner, freshness, estimated expansion cost, and available projections. A ContextView is a bounded projection built for one decision. The step from handle to view is neither retrieval nor expansion — it is materialization: ArtifactHandle → context materialization → ContextView. Teams stop saying "expand this artifact" and start saying "materialize a ContextView."

The ContextView is itself a first-class runtime artifact — content-addressed, in the same model as everything else (id · content_hash · derived_from · planner_version · preview_plan · pins · budget · created_at). So the call path is Mission → ContextView → model call, and the reproducible-working-set is a property of a real object: ContextView = f(handles, preview_plan, pins), replayable. That buys replay, benchmarking, diffing, cache reuse, provenance, inspection, and planner-regression tests for free — you can ask "show me the ContextView that this model saw" instead of reconstructing it.

RuntimeEvent — the normalized envelope every subsystem emits (one forensic timeline):
  event_id · event_type · schema_version · mission_id · session_id · parent_event_id
  actor · capability_id · artifact_handles · source_runtime · source_agent
  timestamp · visibility · policy_context · result_status · evidence_refs

  specializations: DereferenceEvent · CapabilityInvocationEvent · ToolProposalEvent
                   ToolResultEvent · VerificationEvent · MergeEvent · PromotionEvent
                   PolicyDecisionEvent · HumanReviewEvent      (schema spec: v9)

Every dereference is a DereferenceEvent — one member of that family. The rule is normalize first; evaluate and reconstruct second, so the same typed history serves live policy and after-the-fact forensics.

3. Context Preview Plans

Pass-by-reference only works if previews are well designed. Context Runtime emits a plan — Required / Optional / Excluded — before expansion, estimating token cost, retrieval cost, latency, privacy boundary, authority, expected information gain, cache reuse, and context-window risk. Preview strategies (identity-only, structural, change-summary, evidence-balance, unresolved-risk, high-risk diff, historical decision, runtime compatibility, graph neighborhood) are selected by the planner per task, drawing on v7's representation routing rather than re-deriving it.

4. Model-callable harness APIs

The model-facing surface is a set of typed operations — inspect · expand · find_related · dependencies · affected_by · diff · timeline · compare · verify · merge · explain — that return typed handles and bounded views, never text blobs. Each call emits a DereferenceEvent. The division of authority is precise:

The planner decides what should become context. The harness governs how it becomes context.

Models choose among permitted semantic operations; the harness enforces visibility, budgets, type contracts, and provenance; deterministic operators compute graph, diff, dependency, and validation results. A planner regression is a wrong what; a harness regression is an unsafe or wasteful how — separately testable. The full capability-descriptor contract is the v7 amendment; the verify/merge operators are v9.

5. Reference-first retrieval

Retrieval shifts from query → top-k text chunks to intent → candidate handles → preview plan → selective expansion → evidence view. ReDevOps RAG returns exact artifact references, symbols, typed relationships, evidence spans, graph neighborhoods, run diagnostics, decisions, contradictions, and unresolved risks — expanding full content only when the next action requires it. The retrieval-mode and expected-context-value scoring spec lives in the v7 amendment; this paper cites it rather than repeating the representation-routing thesis.

6. Append-only model sessions

A session is an ordered set of additions — a stable system and capability prefix, immutable Mission-state references, then appended observations and decisions — not a prompt whose head is repeatedly rewritten with fresh summaries. This buys provider prompt-cache reuse, traceable context evolution, deterministic replay, less semantic drift, clear supersession, and an honest record of what each call actually knew. Append-only enables cache reuse, but the thesis is architectural, not a caching trick. Because each appended record is content-addressed and the session is hash-chained, the record is tamper-evident: a portable Mission Case Bundle re-verifies it and replay reconstructs the same terminal state, so what a model knew — and what it decided — cannot be edited after the fact.

7. Compaction as governed merge

When session state outgrows the window, compaction is not "ask the model to summarize everything." It is session artifacts → typed merge() → preserved decisions, conflicts, evidence, residual risks → a new compact artifact that retains references to the original evidence and records what was omitted. This is measured. Running naive bounded merge against governed merge() on a fixed 32K integrator over a real corpus — same model, same call count, the only difference being the governance layer — the naive prose digest retained just 14–36% of the corpus entities and worsened with scale, while the governed ledger held ~98%, flat across a 6→23-post, 2→3-level depth ladder. The reference/ledger survives compaction where the serialized prose does not; §7 is the general statement of that result. The operator itself is v9's semantic merge().

8. Scratch state versus durable knowledge

Scratch (temporary hypotheses, local notes, intermediate calculations) is separated from the durable graph (evidence, findings, decisions, assumptions, verified outputs). Promotion requires typed output, provenance, verification, a Mission or Investigation association, and policy approval where required — so free-form agent memory never quietly becomes organizational truth. The same discipline applies to the normalized event stream: RuntimeEvents are raw observation, not knowledge; they may motivate a Finding but never become truth automatically. (Spec: v6 amendment.)

9. Harness strategy as a planned decision

The Execution Planner selects not only a model and tool but a harness strategy — serialized-direct, reference-first, cached append-only session, hierarchical merge, parallel workers, deterministic-only, or human-gated — from context size, dependency density, model capability, artifact authority, budgets, verification coverage, privacy, and failure cost. This widens v8's execution-topology thesis from topology to harness strategy, learned from outcomes.

10. Typed capability surface

Model-callable operations stay governed because each capability declares its contract — inputs, outputs, pre/postconditions, failure modes, side effects, required runtimes, security profile, verification method, and cost/latency estimates. v10 states this only to make §4 governable; the full CapabilityDescriptor spec is the v7 amendment.

11. Security and authorization

Reference-first improves security only if handles are capability-safe: a visibility/tenancy check on every dereference, least-privilege projections, short-lived leases, purpose restrictions, and — critically — no authority inherited merely from a model possessing an identifier, with defenses against reference-substitution and stale-handle attacks. Enforcement is monitor-first: OBSERVE by default, DENY only at clean synchronous gates where the action has not yet occurred and the host can honor the result. A DecisionGate denies only when parsing, rule evaluation, evidence and audit persistence, and correlation state all succeed cleanly — otherwise it returns UNABLE_TO_DECIDE (fail-open or fail-closed declared per surface) rather than rendering an internal failure as an affirmative safety decision. Severity is orthogonal to enforceability, and operator detail (rule ids, evidence) travels a separate ledger, never the model control channel. The full model — trajectory rules, the strict enforcement subset, the coverage matrix — is the security amendment.

12. Failure modes

13. Evaluation

Five arms: full serialization · ordinary RAG · reference-first · reference-first + append-only session · reference-first + governed compaction. Metrics: task success, unsupported claims, evidence coverage, context tokens, cache-hit rate, dereference count, latency, cost, stale-artifact use, authorization violations, replay completeness, compaction information loss. Two metrics nobody else measures become planner objectives (both oriented to [0,1]):

Because the ContextView is a first-class artifact and every reference is a DereferenceEvent, both are computable per call from the trace. Workloads that exceed one context and require cross-artifact reasoning — the merge-overload corpus, DataOpsBench, KYC/property-graph reasoning, and Sidekick software-engineering flows — give v10 the credibility the series earned; the compaction-loss metric is measured across a depth ladder (naive prose 0.14–0.36 and falling vs governed ledger ~0.98 flat).

14. Relationship to the existing stack

The planner decides what becomes context; the harness governs how. The matrix assigns the how.

Componentv10 responsibility
Context Runtimeplans and materializes bounded views
Mission Runtimeowns durable execution state and session lifecycle
Discovery Runtimeproposes new context / investigations from state changes
Execution Plannerselects harness and expansion strategy
ReDevOps RAGfinds handles and evidence projections
merge()compacts or reconciles oversized state
Sidekickexecutes model sessions through the governed harness
Object storeowns referenced artifact + ContextView materialization and leases

15. Implementation roadmap

1. ArtifactHandle + ContextView as first-class artifacts + typed dereference contracts → 2. ContextPreviewPlan (materialization) → 3. read-only graph APIs → 4. reference-first RAG result mode → 5. append-only session records → 6. cache-aware provider adapters → 7. governed compaction via merge() → 8. scratch→durable promotion → 9. planner-selected harness strategy → 10. harness benchmark suite → 11. lease/ownership/security hardening → 12. Sidekick integration. Alongside these, the normalized-event work items: a Runtime Event Contract, a Trajectory Rule engine, the Decision Gate, portable Mission Case Bundles, and an integration coverage matrix.

16. Extension — the Evaluation Runtime

Reference-first context is a general substrate; this extension applies it to one concrete, high-value workload: running AI benchmarks. Nearly every benchmark project independently rebuilds the same nine things — datasets, prompts, harnesses, evaluation loops, metrics, provenance, replay logic, reporting, and publication decisions — each slightly differently. Those are runtime concerns, not evaluation logic. The Evaluation Runtime makes them a reusable, governed substrate, so a benchmark author writes only the part that is theirs (the evaluation) and inherits reproducibility, provenance, replay, verification, and governed publication.

It is infrastructure, not another benchmark, and not a separate runtime — it is the governance layer over the substrate v1–v10 already built. Every benchmark flows through one chain, each stage versioned, reference-first, replayable, and auditable (each emits a RuntimeEvent):

Dataset            versioned · licensed · provenance · limitations · compatibility_hash
   ▼
ContextView        reference-first: only APPROVED handles materialize; reproducible = f(handles, plan, pins)
   ▼
EvaluationProgram  the benchmark AS a versioned Mission program (dataset + protocol + steps + harness)
   ▼   run
Run → Verification → Assessment → Policy → Publication → Finding
       deterministic-first   metrics    clean-decision gate:      derives from evidence;
       (model self-assess     per        unverified is HELD;       traces to the Run + pins
        is never the gate)    protocol   UNABLE_TO_DECIDE->REVIEW  (explicit comparability)

The runtimes compose: Context Runtime builds the deterministic working set; Mission Runtime executes the versioned program; ReDevOps RAG supplies provenance-aware retrieval (handles, not expanded documents); Discovery Runtime keeps benchmarks current — detecting model releases, dataset revisions and restatements and proposing replay Investigations (never auto-republishing); the Evaluation Runtime governs verification, publication and the Claims → Evidence → Runs → Findings → Publication lineage graph, where two findings are comparable only if their dataset and protocol pins match.

It earns its keep by being generic: existing and in-progress benchmarks are all EvaluationPrograms over the same chain, differing only in dataset, protocol, and evaluation logic.

BenchmarkWhat the protocol scoresWhat the substrate provides
DataOpsBenchrouting/decision accuracy operating a data platformpinned incident sets; verified deciders; replay after model releases
KYC / property-graphmulti-hop reasoning + traversal correctness/latencyreference-first graph handles; provenance to source records; comparability across graph revisions
Agent benchmarksgate-pass (compile/tests/architecture) + blinded qualitygoverned harness; deterministic gates; per-pair verdicts as evidence
RAG evaluationsrecall/precision, routing, contradiction handlingreference-first retrieval; hidden retrieval changes made explicit; merge() coverage/conflict metrics
Quantify strategiesscenario outcomes under compatible runtimes + reproducibilityas-of dataset pinning; execution-lag protocol; governed publication

The merge() depth-properties benchmark (§7) is itself a worked example: a fixed integrator, a pinned corpus, seeded contradictions, deterministic coverage and conflict metrics, and an honest publication of both the wins and the ties — exactly the shape the Evaluation Runtime standardizes. The substrate is implemented as an executable specification (agentic_os/mission/evaluation.py), composing the governed merge() operator and the RuntimeEvent family; its self-test verifies reference-first materialization, verified-only publication, lineage, per-stage events, deterministic replay, comparability, and continuous-discovery investigations — 12/12.

What v10 does not claim

Not infinite context; not automatic correctness; not that reference-first is always cheaper; not that graph traversal replaces retrieval; not that models should curate long-term memory freely; not that the agent object becomes the system's primary abstraction. The defensible claim is narrower and stronger:

Reference-first execution turns context construction from prompt assembly into a governed runtime operation. Models operate over governed references to application state; the runtime decides what becomes context, when it is expanded, and how it is preserved.

ReDevOps Whitepaper v10 (draft, for architecture ratification and frontier-model implementation). The harness layer over the three runtimes of v1–v9. One specification (Python), three implementations (Python · Go · Kotlin/Spark). Amendments to v6–v9 and the security/Sidekick docs carry the per-runtime specifications. The Evaluation Runtime extension (§16, above) applies this reference-first substrate to a concrete workload — running AI benchmarks.