LlamaIndex Retrieves. ReDevOps Completes the Closure.
Each integration in this series looks for a framework's strongest native abstraction and tests the boundary immediately outside it, in a fresh experiment. For LlamaIndex the strongest abstraction is unambiguous: retrieval over an index. So that is exactly where we pushed.
We started with feasibility, as always. llama-index-core installs clean into the environment that already holds the Mission SDK, Closure Resolution, nvidia-nat, and pydantic-ai: it only added packages, leaving torch, numpy, and pydantic untouched. The embeddings run locally and offline: we wrapped the shipped bge-small encoder as a LlamaIndex BaseEmbedding, so a real VectorStoreIndex retriever and query engine run in-process with no external service. No mock, no sidecar.
Similar is not complete
The flagship uses no LLM, so the mechanism is unambiguous. We index 1,138 real symbols from a pinned code tree and give a real LlamaIndex retriever five refactor tasks. Each task has a frozen oracle: the seed symbol being changed, and the full closure a correct change must touch, including the contract callers, the functions whose behaviour breaks if the change lands without updating them.
The retriever does what retrieval does well. It finds the seed and the type/schema edges, because those read like the change description. It does not find the contract callers, because a caller three hops up a call chain shares almost no vocabulary with the change: it is structurally linked and textually dissimilar. Averaged over the tasks, the retriever alone recovers closure recall 0.320 and contract-caller recall 0.300.
ReDevOps does not replace the retriever. It registers it as one representation and composes it with the static call-graph closure: take the structural must-change members first, then let the LlamaIndex retriever fill the remaining budget. Same token budget, same accounting. Closure recall rises to 0.693 and contract-caller recall to 0.600, roughly double, and on the one task in five where semantic retrieval already resolves the closure, composition simply preserves it. Zero regressions. The retriever is preferred when it suffices, composed when it does not.
One honesty note we build into the numbers: the composed arm is structurally seeded on the symbols being changed, a different input than the retriever's natural-language query. That makes its match recall high by construction, so we do not report match recall as a retrieval win. The meaningful deltas are the two the seeding does not hand you for free: closure and contract-caller recall.
The query engine always answers
The second experiment brings back the LLM and uses LlamaIndex's abstraction beyond retrieval: the query engine, which synthesizes a fluent answer from whatever the retriever returned. We buried a synthetic needle ("the access code for facility X is …") in a real LiveRAG haystack and asked the query engine for the code.
When the retriever finds the needle, the query engine is right, every time. When retrieval misses it, the query engine is confidently wrong, and, crucially, it still answers. On the misses, four of six came back with a fabricated code stated as plainly as the correct ones. That is the risk just outside the query engine: synthesis fluency is independent of whether the evidence is actually there.
ReDevOps adds the signal the synthesizer does not: it checks whether the answer's value is actually present in the retrieved evidence, and abstains (REQUIRE_REVIEW) when it is not. On all six unsupported answers, verification abstained. The result is a selective accuracy of 1.000 at 50% coverage: answer when the evidence supports it, route the rest to a human, and never emit a confident fabrication as if it were a fact.
Retrieval informs; the runtime governs
The last experiment is about behaviour, separate from recall. A retrieved-evidence change is consequential (it writes to a repository), so it runs only through a Mission. No authority grant ⇒ zero unauthorized writes. Grant but no approval ⇒ parked at a human gate, zero side effects. Approved ⇒ it fires exactly once. Replay of the sealed run ⇒ zero duplicate side effects, reconstructed from the event ledger, with no re-execution. Good retrieval is not authority to act, and the two concerns stay cleanly separated: LlamaIndex owns finding the evidence, ReDevOps owns whether the action is allowed to fire.
The full LI-A…LI-E result matrix, the frozen environment, and the acceptance-bundle hashes are on the benchmark page, and the code (a real llama-index retriever and query engine, one Python env) is on GitHub. Same provider-neutral contract as the NVIDIA and PydanticAI slices; new framework support is a conformance exercise on the existing architecture.