redevops-rag

8 slides · Context Runtime's reference retriever

REDEVOPS-RAG1/8

A library, not a framework or a service

  • Small, pip-installable hybrid retrieval with a three-line API and a first-class CLI.
  • No cluster, no service mesh — index and search from one process.
  • Serves as Context Runtime's reference Retriever/Store binding — one source of truth per artifact, retrieval normalized like a database, not copied.
  • One of the engines the runtime routes to, not a competing stack.
REDEVOPS-RAG2/8

Hybrid search in one DuckDB file

  • Dense retrieval: BGE-small sentence embeddings by default (Nemotron-3-Embed and ReasonIR encoders are selectable) with DuckDB cosine similarity.
  • Sparse retrieval: DuckDB FTS BM25 for exact keyword precision.
  • Dense and sparse live in a single file — no multi-service stack to operate.
  • Deterministic ingestion: chunk ids are content-addressed ({document_ref}::{content_hash[:16]}), not position-based — identical content yields identical ids regardless of ingest order, and re-ingest is orphan-safe (a content_hash column plus a prune sweep, so no stale vector survives an edited or shrunk source). Provably reproducible and contamination-immune.
REDEVOPS-RAG3/8

Reciprocal Rank Fusion with priors

  • RRF fuses lexical precision with semantic recall — no score-scale tuning required.
  • Recency and keyword priors weight the fusion toward fresher, more trusted evidence.
  • An optional BAAI cross-encoder reranks the top-k for the queries that matter most.
  • Search returns the chunks with their RRF, boosted and rerank scores; in the Context Runtime binding those hits become content-addressed ArtifactHandle references with provenance and the ranking evidence that surfaced them — a ContextView is materialized on demand, never pasted text.
querydenseBGE + cosineBM25keywordRRF fusereciprocal rankrerank → top-kcross-encoder
REDEVOPS-RAG4/8

document_ids pre-filtering

  • Retrieval can be scoped to an explicit set of document ids before ranking.
  • Tenant, project, or permission boundaries are enforced at read time, not after.
  • Keeps multi-tenant corpora isolated without a separate index per tenant.
  • Parity between the DuckDB store and the pgvector store (pg_store.py, shipped in v0.3.0) — both take document_ids.
REDEVOPS-RAG5/8

Routed by query shape, not forced through one index

  • Structured/analytical questions belong on the DB's own retriever, not a vector index.
  • Multi-hop questions route to a graph arm; temporal questions to a time-aware arm.
  • redevops-rag is the dense+sparse arm the Context Runtime router selects for semantic recall.
  • For change-closure tasks a structural arm complements content retrieval: citation_graph (ChangeClosureGraph, shipped in redevops-rag v0.4.0) retrieves the reverse dependency closure of an edit and union_retrieve fills the remaining token budget with hybrid content hits — benchmarks show the two miss different edges, so their union recovers the change closure (the definition plus every caller whose contract breaks), and that retrieval lift converts into a successfully applied edit. The kernel's separate code_graph retriever covers code definitions and references. write-up →
  • The right regime per query beats one pipeline for all queries.
REDEVOPS-RAG6/8

Calibrated relevance and abstention

  • In the Context Runtime layer, redevops-rag scores are calibrated (isotonic) to probabilities, so low-confidence reads can abstain instead of guessing; redevops-rag itself returns the raw ranked scores.
  • Grounded P(relevant) makes 'served vs declined' an auditable decision.
  • Abstention is a feature: a clean 'I don't know' beats a confident wrong answer.
  • Inspect the ranking and the decline reasons at redevops.io/planner.
REDEVOPS-RAG7/8

Benchmark-honest, not benchmark-tuned

  • Measured per query shape rather than a single blended score.
  • On MuSiQue multi-hop, reasoning retrieval (DIVER 0.93) and HippoRAG (0.73, Personalized PageRank over its fact graph) pull clear of BM25 and hybrid (0.60), while the dependency-free graph-similarity arm trails (0.20); the routed policy lands 0.97.
  • On temporal conversational memory (LongMemEval) DIVER leads (0.89 vs 0.34 for graph-similarity); on TEMPO every method ties at retrieval (0.59), so that gap is on the answer side. The router encodes those findings.
  • Results and methodology are published at redevops.io/benchmarks/retrieval.
REDEVOPS-RAG8/8

Local hybrid retrieval, three lines

  • pip install, index a corpus, search — one Python call or one CLI command.
  • DuckDB handles dense vectors and BM25 in one file; zero cloud sync.
  • Reciprocal Rank Fusion with optional cross-encoder reranking.
  • The dependency-light default that the Context Runtime binds and routes to.