The Context Wars, Part II: The End of RAG?

Everyone assumed Retrieval-Augmented Generation was the future.
It was the cheapest workaround we had. It solved yesterday's problem.
In Part I we saw why RAG became the default: long contexts were too expensive, so we taught a model to retrieve. It worked remarkably well. Until it didn't.
The retrieval paradox
Every RAG system has one job: predict what the model will need before it starts reasoning.
Ask 'Explain why deployment X failed.' The retrieval engine can't know whether the answer lives in Kubernetes logs, Terraform, Slack conversations, Git history, Grafana dashboards or internal documentation. It guesses. And sometimes the missing paragraph is the only one that mattered.
This is why production RAG became increasingly sophisticated.
Hybrid search. BM25. Reciprocal Rank Fusion. Cross-encoder rerankers. Parent-child chunking. Knowledge graphs.
Every improvement answers the same question: how do we retrieve exactly what the model will need? And every one of those techniques lives outside the model. The model itself hasn't changed. We keep building more elaborate retrieval to compensate for the limits of attention.
DeepSeek asks a different question
DeepSeek asks 'What if the model didn't need retrieval to be nearly as precise?' A different philosophy: compress information inside the model.
Its hybrid attention architecture combines Compressed Sparse Attention (CSA), Heavily Compressed Attention (HCA) and sliding-window attention. CSA dynamically prioritizes the context that matters for the current query. HCA aggressively compresses older context into compact global representations. Sliding-window attention keeps recent tokens at full fidelity while older information is progressively summarized.
Classic RAG says: read less. DeepSeek says: remember more efficiently. One moves complexity outside the model, the other inside.
Did DeepSeek eliminate RAG?
No. Retrieval still solves problems attention never can.
Private company documents. Fresh information. Dynamic databases. APIs. Real-time events.
No amount of compressed attention lets a model remember documents it has never seen. Retrieval doesn't disappear; its role shrinks: from compensating for weak attention to bridging the model and external knowledge. As attention improves, developers spend less time fighting context and more time building systems around reasoning.
DeepSeek changes the mathematics of attention. But another lab appears to be asking an entirely different question: what if one model never had to read everything in the first place?
That brings us to Part III.

