Access-controlled RAG

A shared knowledge base holds three tenants' documents. A user asks about their order — but retrieval ranks by relevance, and relevance doesn't know about permissions, so other tenants' chunks routinely land in the top-k. From that moment you're negotiating with probability. The question this page measures: where do you enforce privacy, and what does each choice actually stop?

retrieval filteroutput screen prompt instructionmulti-tenant

Four places to enforce, one that works by construction

600 simulated queries against the shared index — a quarter of them deliberately fishing for other tenants' data, the rest ordinary. The pipeline below highlights where your chosen control sits; the bars count what actually leaked:

Where is privacy enforced?

enforcement point

top-k retrieved more chunks = more chances for a foreign one

share of fishing queries deliberate probes vs ordinary questions

The numbers make the argument better than any policy document. With no control, contamination is nearly universal and leaks follow. The prompt instruction genuinely helps — and still leaks dozens of times per 600 queries, because an instruction is followed probabilistically and the fishing queries are engineered to talk it out of compliance. The output filter catches most of what the prompt misses and still lets paraphrases through: it is scanning for content it can recognise, and language has infinitely many ways to say the same fact. Only the retrieval filter reports zero — not because it tries harder, but because the foreign chunks were removed before the model existed in the story. What never enters the context cannot leak, at any temperature, under any prompt.

# the load-bearing line in every multi-tenant RAG system
chunks = index.search(query, k=5,
                      filter={"tenant_id": current_user.tenant_id})   # BEFORE the model
# an equivalent design: one index per tenant — same invariant, different mechanics

Layering still matters: the retrieval filter enforces tenancy, but the output screen still earns its keep against a different threat (the model repeating internal notes that legitimately are in context), and the prompt still shapes tone and refusal behaviour. The pro-level claim is not "prompts are useless" — it is that each control has a failure class it cannot address, and tenancy's failure class is only closed at retrieval.

⚠️ Traps & honesty: the compliance rates in this simulation (a prompted model reveals foreign context 12% of the time under fishing, 2% otherwise; the output filter catches 70%) are assumptions of the toy, not measurements of any real model — the structural conclusion (retrieval filtering is zero by construction; everything else is a probability) is what transfers · "relevance doesn't know about permissions" assumes a shared index; per-tenant indexes make the same guarantee with more operational overhead · a retrieval filter is only as correct as the ACL metadata on the chunks — mislabelled documents leak through any architecture.

Takeaways: retrieval ranks by relevance, and relevance ignores permissions — so in a shared index, foreign chunks reaching the context is the default, not the anomaly · a prompt instruction reduces leaks and cannot eliminate them; an output filter catches recognisable content and misses paraphrases · the ACL filter at retrieval is the only control that is zero by construction — enforce tenancy where the data enters, not where it exits · keep the other layers for the failure classes they do close. Next: A/B testing mechanics.

Second opinion (taught here — these corroborate): Anthropic docs · embeddings & retrieval · Building effective agents.