All articles
    rag
    context-window
    llm-optimization
    retrieval-augmented-generation

    Your RAG Is Not Failing. It Is Reading Too Much.

    July 6, 2026redevops.io
    Your RAG Is Not Failing. It Is Reading Too Much.

    Most RAG failures are context pollution failures. Retrieval still works.

    The correct document is often retrieved, but the model also receives unrelated documents, stale notes, wrong-domain records, old decisions, and tool output that never belonged in the context window. That is how systems produce confident wrong answers while dashboards still look green.

    The failure mode

    A user asks:

    What medications am I currently taking?

    The retriever finds the right medical record, but it also pulls financial documents because they contain words like:

    • statement
    • balance
    • discharge
    • liability

    The answer is present; the context is contaminated. Most RAG evals still count that as a retrieval success because the correct source appears somewhere in top-k. And the user sees a wrong answer.

    What we measured

    On a mixed Financial + Medical dataset:

    StrategyRecallCross-domain noise
    Flat mixed indexpreserved2.5 wrong-domain docs/query
    Sharded + Hybrid RRFpreserved2.9 wrong-domain docs/query
    Sharded + learned routingpreserved0.0 wrong-domain docs/query

    The surprising result: hybrid fusion made irrelevant context worse. The answer was findable and polluted.

    Why this matters

    Diagram showing correct retrieval vs context bloat
    Scroll the diagram sideways to read itFull size

    Most systems optimize:

    Can I retrieve the correct document?

    Production systems need to optimize:

    What should the model not read?

    Those are different problems.

    This gets worse as teams add more sources, tools, MCP servers, retrievers, memories, and skills. Capability count grows, context quality falls, and the model sees more definitions, schemas, snippets, and partially relevant evidence than it needs. That is context bloat.

    The remedy

    Diagram showing planner filtering tools before sending context to model
    Scroll the diagram sideways to read itFull size

    Stop injecting every capability into the model context. Treat capabilities as plugins selected by the planner, with tools in the prompt only when selected. The model should not see every retriever, database schema, MCP tool, skill, and memory store; the planner should select the minimum evidence needed.

    For Context Runtime, that means:

    • retrieval methods live in a registry
    • data sources live behind plugins
    • memory indices are selected per request
    • tool schemas stay out of the prompt unless needed
    • the planner chooses the execution strategy before the model is called

    The result is cleaner context, lower cost, and fewer confident wrong answers.

    Bigger windows do not solve context pollution. Better planning solves it.


    Reference implementation: https://github.com/redevops-io/context-runtime
    Benchmarks: https://redevops.io/benchmarks
    Whitepaper v2: https://redevops.io/whitepaper-v2


    redevops.io builds rAIlsApps (source-available (AGPL-3.0 with the Commons Clause)), self-hostable modules run by agents, on hardware you can actually afford. See it run.

    Want ReDevOps in your Google Top Stories & AI results?Follow ReDevOps on Google