Privacy-first LLM gateway for secure AI prompts

An LLM gateway is a routing layer that accepts AI requests from your application and forwards them to one or more model providers. It is where authentication, model selection, rate limits and observability live. In Privian's case, it is also where PII masking and prompt privacy run before any provider call. See the glossary entry for a longer definition, or the AI Security Layer category page for how it fits into the broader stack.

A pure routing gateway changes nothing about what the provider sees. Customer emails, internal documents, support transcripts and developer scratch prompts all reach the model verbatim. A privacy-first gateway shrinks that exposure surface at the edge.

• Customer identifiers in product AI features
• Employee data in internal copilots
• Ticket content in customer support workflows
• Secrets and API keys in developer experimentation

1. Step 1

   Client → Privian

   Your app POSTs prompt + model to a single Privian endpoint with a gateway API key.

2. Step 2

   Detection

   Inbound prompt is scanned for personal and sensitive entities.

3. Step 3

   Masking

   Detected values are replaced with deterministic placeholders (PERSON_1, EMAIL_2, ...) for the request.

4. Step 4

   Provider call

   Masked prompt is forwarded to the configured provider using your decrypted BYOK credentials.

5. Step 5

   Rehydration

   The provider response is scanned and placeholders are restored from the in-memory mapping.

6. Step 6

   Response

   Your app receives a normal, rehydrated response. The mapping is discarded.

The beta exposes a single gateway endpoint with a small JSON body.

curl https://api.privian.io/v1/gateway \
  -H "Authorization: Bearer $PRIVIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.5",
    "prompt": "Email me at michael@example.com about ticket #4821.",
    "stream": false
  }'

Provider-namespaced model ids. No messages[], no system prompt array, no tool definitions in the beta. See the API reference for the full request and response shape.

• Multi-provider routing

  OpenAI, Anthropic and other providers behind one endpoint.

• PII masking

  Personal and secret entities replaced before the provider call.

• Rehydration

  Placeholders restored in the response so your app sees real values.

• BYOK

  Your provider credentials, encrypted at rest, used at request time.

• Gateway API keys

  Scoped Privian keys (sk-gw_live_… / sk-gw_test_…) for your apps.

• Structural observability

  Counters and metadata without storing raw prompts.

• No OpenAI SDK drop-in compatibility
• No messages[] / chat array support
• No tool / function calling
• No JSON mode
• No native provider token streaming — stream: true returns artificially chunked text
• No HIPAA / SOC 2 / PCI claims at this time

What is an LLM gateway?

An LLM gateway is a routing layer that accepts AI requests from your application and forwards them to one or more model providers. It is where authentication, model selection and observability live. Privian adds PII masking, prompt privacy and rehydration on top of that routing layer.

How is Privian different from a generic AI gateway?

A generic AI gateway just routes traffic — whatever you POST, the provider sees. Privian is privacy-first: sensitive values are detected and replaced with deterministic placeholders before the provider call, then restored in the response.

Does Privian support the OpenAI SDK as a drop-in?

Not in the current beta. The gateway exposes a small JSON contract — POST prompt, model and an optional stream flag. The OpenAI messages[] schema, tool/function calling and JSON mode are not exposed yet.

Does the LLM provider see personal data?

No. Detected entities are masked at the edge with stable placeholders. Only the masked prompt reaches the provider.

How does BYOK work?

You add your provider credentials (OpenAI, Anthropic, etc.) in the dashboard. They are encrypted at rest using AES-GCM and decrypted in-process at request time. Token usage continues to bill against your provider account — Privian does not resell tokens.

Does the gateway store prompts or responses?

No. Raw prompts and responses are never persisted. Only structural counters are kept for observability.

• Category: AI Security Layer
• Glossary: What is an LLM gateway?
• Concepts: BYOK
• Authentication
• Quickstart
• Architecture overview
• API reference
• PII masking