How to route prompts securely

The patterns that actually work

Secure routing is not one big decision. It is a small set of consistent choices that compound:

1. One front door — every prompt goes through the same gateway
2. One credential per provider, held server-side
3. One naming scheme for models, validated at the edge
4. One audit trail, with metadata only

One front door

The single biggest reduction in operational risk is making "where do prompts go" have exactly one answer. With Privian that answer is POST /v1/gateway on https://api.privian.io.

Provider credentials

Two patterns work. Both keep the credential off client machines:

• Pooled credentials. The gateway holds its own provider keys and you authenticate only with the gateway API key.
• BYOK (bring your own key). You upload your provider key once; the gateway encrypts it at rest with AES-GCM and uses it for your requests only. See BYOK.

Model identification

Model IDs are namespaced by provider:

openai/gpt-5.5
openai/gpt-5.5-mini
anthropic/claude-sonnet-4.5
google/gemini-3.5-flash

The gateway resolves the provider from the prefix, applies the right credential, and rejects unknown models before any upstream call. This stops a caller from probing arbitrary providers by crafting model names.

Audit without archive

Every routing decision should be visible after the fact — which model, which provider, how long it took, how many entities were masked. None of that requires storing the prompt body. See Zero retention.

A worked example

curl https://api.privian.io/v1/gateway \
  -H "Authorization: Bearer $PRIVIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "prompt": "Draft a reply to jane@example.com",
    "stream": false
  }'

Same gateway, same shape of request, different model — the routing happens inside the gateway. Your application code does not change.

How Privian fits

Privian provides the single front door, the credential management (pooled or BYOK), the model registry, and the audit trail. See LLM Gateway for the product detail.