Your retriever and vector store
Retained per your own indexing and storage policy — entirely outside Privian's visibility or control.
Solution
RAG privacy: the question is safe, the retrieved context isn't
Secure RAG architecture for teams shipping retrieval-augmented generation.
In most RAG incidents, the user typed nothing sensitive. The retrieved documents did the damage — pasted into the prompt, in full, by a retriever that was never asked to evaluate sensitivity. Mask the assembled prompt at the generation hop, after retrieval, without touching your retriever.
How do you keep personal data out of RAG prompts?
Retrieved chunks are assembled into the prompt unreviewed, so the practical control is at the generation hop: scan the fully assembled prompt — question plus retrieved context — for supported identifiers, mask them with deterministic placeholders before the call to the model provider, and restore the originals in the response. This is complementary to, not a replacement for, access control and redaction earlier in the pipeline.
Summary
The short version
What is the problem?
RAG prompts carry identifiers from documents nobody reviewed at query time — the user's question is usually clean, but the retrieved context is not, and it is assembled into the prompt at volume on every request.
How is it solved?
Mask the assembled prompt — question plus retrieved chunks — at the generation hop, after retrieval has finished, then rehydrate the response so the application still sees real values.
When should this approach be used?
When your RAG pipeline retrieves from a document set that was never classified for sensitivity, when embedding or generation providers receive full chunk content, or when a security review asks what a retrieved document exposes downstream.
What are the limitations?
Masking is best-effort over a supported entity set, applies only at the generation call, and does not evaluate whether a document should have been retrieved, embedded, or indexed in the first place.
Problem
Where privacy breaks in a RAG pipeline
A RAG pipeline has more hops than a simple chat completion, and each one has a distinct privacy exposure. The user's question is rarely the risk — it is short, and typed with intent. Everything after retrieval is content nobody in the request path chose to disclose.
Framework
Hop-by-hop exposure in a typical RAG pipeline
- 01
Ingest
Documents enter the index as-is. Whatever sensitivity they contain enters with them, usually without per-document review.
- 02
Chunk
Splitting documents into passages does not remove identifiers — it just distributes them across more retrievable units.
- 03
Embed
Chunks are typically sent in full, unmasked, to a hosted embedding provider before a vector is ever stored.
- 04
Vector store
Stored vectors and their associated text sit at rest with whatever access control the store enforces — or doesn't.
- 05
Retrieve
A query pulls the top-k chunks by similarity, not by sensitivity. Relevance ranking has no concept of PII.
- 06
Assemble prompt
Retrieved chunks are concatenated into the prompt verbatim. This is where identifiers from documents, not from the user, enter the request.
- 07
Generate
The assembled prompt — question plus context — is sent to the model provider as one request.
- 08
Response
The model's answer may echo back identifiers it was given in context, extending exposure into the output.
- 09
Logs
Prompt logging, tracing and analytics frequently persist the full assembled prompt, including retrieved content, long after the request completes.
Three of these deserve particular attention because they are easy to miss: embedding-time exposure to a hosted provider, over-retrieval that pulls more of a document than the query needed, and stale or over-permissive index entries that keep surfacing content that should have been removed or restricted.
Access control at the retrieval layer answers "who may see this document." It does not answer "what leaves our boundary when this document is pasted into a prompt."
Common confusion
Three controls that get conflated — and aren't the same tool
Teams often reach for one of these and assume it covers the other two. Each answers a different question and fails differently when asked to cover for the others.
Framework
Retrieval access control vs. ingest redaction vs. egress masking
- 01
Retrieval access control
Governs who may retrieve a document at all. Right tool when the concern is authorization — e.g. a customer's data should never surface in another customer's retrieval results. Does nothing about what a document's content exposes once it is legitimately retrieved.
- 02
Document-level redaction at ingest
Removes or masks identifiers before a document enters the index. Right tool when you control the ingest pipeline and can afford to re-run it as documents change. Does not help with documents indexed before redaction existed, or with content added outside that pipeline.
- 03
Egress masking at generation
Masks the assembled prompt — including retrieved content — immediately before the provider call, regardless of what the index contains or how it got there. Right tool as a backstop that applies uniformly across every retriever and every document, including ones nobody redacted.
These are complementary, not competing. A defensible RAG pipeline typically uses access control to gate retrieval, redaction to reduce what enters the index, and egress masking as the control that catches whatever the first two missed.
Scope
What Privian covers in a RAG pipeline — and what it doesn't
Privian sits at exactly one hop: the call from your application to the LLM provider, after your retriever has already run. It does not index, embed, chunk or retrieve anything, and it has no visibility into your vector store, your ranking logic or your document permissions.
In scope for Privian
- Scanning the assembled prompt — question and retrieved context together — for the supported entity set
- Masking detected identifiers with deterministic placeholders before the generation call
- Rehydrating the response so the application sees real values
- Doing this regardless of which retriever or vector store produced the context
Remains the customer's responsibility
- Access control on who may retrieve which documents
- Document-level redaction at ingest, if used
- What is sent to a hosted embedding provider
- Chunking strategy, index freshness, and removing stale or over-permissive entries
- Retrieval relevance and ranking — Privian does not judge whether a chunk should have been retrieved
This division of responsibility is deliberate: a gateway that also tried to govern ingest and retrieval would need to understand your document permissions, your chunking strategy and your embedding provider — none of which it can safely infer from a single API request.
Provenance
About RAG and LLM risk claims
Example
Masking assembled context, not the question
The identifiers below did not come from the user's question — they came from the retrieved chunk the application assembled into the prompt for context.
Assembled prompt sent to Privian
Question: What's the status of this
customer's onboarding?
Retrieved context:
"...call notes from Priya Nair
(priya.nair@acmecorp.com,
+1-415-555-0199) regarding
account ACC-88213, escalated
after a failed payment on card
ending 4242..."Privian sends to the provider
Question: What's the status of this
customer's onboarding?
Retrieved context:
"...call notes from PERSON_1
(EMAIL_1,
PHONE_1) regarding
account ACC-88213, escalated
after a failed payment on card
ending CREDIT_CARD_1..."The question was clean to begin with. Masking still ran on it — the whole assembled prompt is scanned every time — but the identifiers it caught all came from the retrieved passage.
Mechanics
Rehydration and retention per hop
The placeholder mapping for a request — including any identifiers pulled from retrieved chunks — is held in memory only for the life of that request. It is applied in reverse to the provider's response and then discarded; unknown placeholders in a response pass through untouched. See rehydration for the mechanics.
Your application logs
If retrieved context or assembled prompts are logged upstream of Privian, that retention is governed by your logging configuration, not by the gateway.
Privian
Does not persist raw prompts or responses, including the retrieved context portion. Only structural counters are retained for billing and observability.
The model provider
Governed by the provider's own terms and your configuration for that account — check the provider's current documentation.
Architecture
Where Privian sits in a RAG request
- 1
Retriever — Your vector store returns the top-k chunks for the query, using your existing ranking logic — unchanged.
- 2
Application assembles prompt — Your code concatenates the question and retrieved chunks into the final prompt, exactly as it does today.
- 3
Privian masks — The assembled prompt is scanned for the supported entity set and detected values are replaced with deterministic placeholders.
- 4
Provider — The masked request is forwarded to the model provider using your own encrypted credentials.
- 5
Rehydrate — Placeholders in the response are replaced with original values from the in-memory mapping, which is then discarded.
- 6
Application — Your code receives an ordinary provider-shaped response with real values — no retriever or prompt-assembly changes required.
Checklist
RAG security review checklist
Framework
Questions to answer before a RAG feature ships
- 01
Who can retrieve which documents?
Access control at the retrieval layer, verified per index and per user role — not assumed from application-level auth.
- 02
What was reviewed for sensitivity before ingest?
If nothing was, assume retrieved chunks may contain identifiers and plan for a control downstream of retrieval.
- 03
What does your embedding provider receive?
Chunk content is usually sent to it in full and unmasked — evaluate that hop separately from generation.
- 04
What leaves the boundary at generation?
The assembled prompt, including retrieved context — this is the hop egress masking addresses.
- 05
Can a retrieved document surface identifiers unrelated to the query?
Over-retrieval can pull more of a document than the question needed, exposing content the query never asked about.
- 06
How stale is the index?
Documents removed or reclassified upstream may still be retrievable if the index wasn't updated with them.
- 07
What do prompt logs and traces retain?
Full assembled prompts, including retrieved content, are a common and often-overlooked retention point.
- 08
Can the response echo identifiers back to the user?
A model can restate context it was given — worth checking who receives the final answer.
Fit
When this pattern is useful — and when it isn't
Useful when
- Your RAG pipeline retrieves from documents that were never classified for sensitivity
- You want a control that applies regardless of retriever, vector store or chunking strategy
- A security review is asking what a retrieved document exposes to your model provider
- You want ingest and retrieval to stay unchanged while adding a generation-hop backstop
Less useful when
- Your actual gap is retrieval authorization — the wrong user seeing the wrong document
- You need control over what your embedding provider receives
- You need native streaming, tool calling or multimodal RAG input today
- Your requirement is a certification rather than a technical control
Scope
What this does NOT solve
FAQ
Frequently asked questions
- How do you keep personal data out of RAG prompts?
- Retrieved chunks are assembled into the prompt without ever being reviewed for sensitivity, so the practical control is at the generation hop: scan the fully assembled prompt — question plus retrieved context — for supported identifiers, mask them with deterministic placeholders before the call to the model provider, and rehydrate the response. This does not replace access control or redaction earlier in the pipeline; it catches what reaches the model regardless of what happened upstream.
- Isn't retrieval access control enough to keep RAG safe?
- Access control answers who may retrieve a document. It does not answer what happens once that document's content is pasted into a prompt and sent to a model provider. A user with legitimate access to a document can still cause its contents — including identifiers unrelated to their query — to leave your boundary on every request that retrieves it.
- Does masking break retrieval relevance?
- No. Privian sits after retrieval, at the generation hop. Retrieval, ranking and chunk selection all happen against your unmasked index exactly as they do today; masking is applied to the assembled prompt only after retrieval has already decided what to include.
- Should I redact at ingest or mask at egress?
- They solve different problems. Ingest-time redaction reduces what ever enters your index, which is valuable but has to be re-run whenever documents change and cannot fix documents already indexed without it. Egress masking controls what leaves your boundary on each request regardless of what the index contains, including content that was never reviewed at ingest. Most RAG deployments need both, not one instead of the other.
- Does sending chunks to a hosted embedding provider expose the same data?
- It can. Embedding is a separate hop from generation, and chunks are typically sent to the embedding provider in full, unmasked, before a vector store ever sees them. Privian does not sit at this hop — it operates at the generation call. Embedding-time exposure is a customer-side architecture decision to evaluate separately.
- Does Privian control what my retriever returns?
- No. Privian has no visibility into your retriever, your vector store or your ranking logic. It receives whatever prompt your application assembles after retrieval has already happened, and applies masking to that assembled text.
- What happens to over-retrieved or stale document content in the prompt?
- If a chunk is retrieved and assembled into the prompt, Privian will scan it like the rest of the prompt and mask any supported identifiers it contains. It cannot tell you that the chunk should not have been retrieved in the first place — that is a retrieval-relevance and access-control question, not a masking question.
- Is this a substitute for document-level access control in my vector store?
- No. If a document should not be retrievable by a given user or query, that has to be enforced at the retrieval layer itself. Masking reduces what a retrieved document exposes to the model provider; it does not restrict which documents get retrieved.
- What does Privian actually cover in a RAG pipeline?
- The generation hop only: the call from your application to the LLM provider, after your retriever has already selected and assembled context. Privian is OpenAI Chat Completions-compatible for non-streaming requests, using your own provider credentials (BYOK, AES-GCM encrypted at rest, decrypted in-process at request time). It does not index, embed, chunk or retrieve anything.
Enterprise review
Built for enterprise review
Trust assets procurement and security teams routinely request.
Plans & pricing
Pricing for masking at the generation hop
BYOK, zero retention and masking of the assembled prompt before the provider call. Beta plans may change as the product matures.