Can You Explain Why Your AI Read That Document?

Ask an AI system a question. Then ask the real question:
Why did it read those documents?
Most systems cannot answer. They can show citations and retrieved chunks. They cannot explain the decision that selected them.
That is the problem.
Citations are not explainability
A citation tells you what the model used.
It does not tell you:
- why that source was retrieved
- why another source was rejected
- why this retrieval strategy won
- whether the score was reliable
- whether the source was allowed
- whether the model should have abstained
As retrieval gets more complex, this gets worse. Hybrid search. Graph retrieval. Reranking. Memory. MCP tools. Long context. Multiple data sources.
More machinery means more decisions, and more to observe.
The enterprise problem
For internal systems, retrieval is relevance, and it is permissioning.
A user may access:
- this database
- not that database
- this table
- not that column
- this document
- not that folder
- these rows
- not another user's rows
If permissions are checked after retrieval, it is too late. The planner should only consider feasible plans, and policy should define the execution space before optimization begins.
The remedy
Treat access control as part of planning.
A candidate plan is valid only if it satisfies:
- user permissions
- tenant isolation
- data-source ACLs
- row/column policies
- budget limits
- provider restrictions
- verification requirements
Then optimize only inside that allowed space.
This is RBAC, and it is policy-constrained planning.
The point
Production AI needs to answer two questions:
- Why did the model read this?
- Was it allowed to?
If the system cannot answer both, it is not ready for enterprise use.
Reference implementation: https://github.com/redevops-io/context-runtime
Benchmarks: https://redevops.io/benchmarks
Whitepaper v2: https://redevops.io/whitepaper-v2
redevops.io builds rAIlsApps (source-available (AGPL-3.0 with the Commons Clause)): self-hostable modules run by agents, on hardware you can actually afford. See it run.