Does self-hosting an LLM solve this?

Self-hosting changes where the data goes — it no longer leaves your infrastructure. It does not, by itself, decide what data is allowed in the prompt. Many self-hosted deployments still benefit from prompt-level data protection internally.

Acceptable-use policies set expectations but cannot intercept a prompt mid-flight. Policy works best alongside technical controls, not in place of them.

Does BYOK stop the provider from seeing data?

BYOK changes the account and billing relationship but not what the provider receives. The model still sees the prompt content. Prompt-level data protection is the layer that changes what the model receives.

How to stop LLMs from seeing sensitive data

Why this question is hard

Useful prompts tend to be rich. A support assistant needs the ticket. A coding copilot needs the snippet. A summarization feature needs the source document. The same characteristics that make a prompt useful also make it likely to contain regulated values.

The available approaches

Framework

Approaches and what each one changes

01
Policy
Set expectations through acceptable-use rules and training. Necessary, but cannot intercept a prompt mid-flight.
02
Application minimization
Drop fields the prompt does not need before composing it. The largest single reduction, cheapest to implement.
03
Prompt-level masking
Detect supported sensitive entities and replace them with deterministic placeholders before egress.
04
Redaction
Destroy values that the model never needs. Stronger than masking but irreversible.
05
BYOK and provider controls
Change the account boundary and provider-side retention. Does not change what the model receives.
06
Self-hosted inference
Move where the model runs, not what it receives. Often combined with prompt-level controls internally.

Prompt Privacy flow— Sensitive values are masked before egress and rehydrated on the return path.

Tradeoffs

Latency. Detection and masking add a small amount of latency at the gateway — typically a few milliseconds.
Recall. No detector catches every sensitive value. Anything outside the supported entity set reaches the provider unchanged.
Capability. Aggressive masking gives the model less context. For most workloads, deterministic placeholders preserve enough structure for the model to reason well; for some, the raw value is genuinely required.
Operational complexity. Self-hosting is the most invasive change. Prompt-level controls are usually the least invasive.

Honest limitations

Prompt-level data protection is one layer, not a complete answer. It does not prevent prompt injection, content-policy violations or downstream misuse. It does not certify HIPAA, SOC 2 or PCI compliance. It does not replace governance decisions about which tools are permitted.

It does, reliably, reduce what an external model receives. That is usually the highest-leverage move available to a team already shipping AI features.

Where to start

Start with application minimization — drop fields the prompt does not need. Then put a gateway in front of the model providers so masking, BYOK and retention controls all live at a single control point. See the Prompt Privacy pillar for the category map and Prompt Security for the implementation surface.

Written under our editorial principles: implementation-grounded, honest about limitations, educational first.

Frequently asked questions

Does self-hosting an LLM solve this?: Self-hosting changes where the data goes — it no longer leaves your infrastructure. It does not, by itself, decide what data is allowed in the prompt. Many self-hosted deployments still benefit from prompt-level data protection internally.
Is policy enough?: Acceptable-use policies set expectations but cannot intercept a prompt mid-flight. Policy works best alongside technical controls, not in place of them.
Does BYOK stop the provider from seeing data?: BYOK changes the account and billing relationship but not what the provider receives. The model still sees the prompt content. Prompt-level data protection is the layer that changes what the model receives.