How to secure prompts before they reach GPT

What "before GPT" means

The most leverage you have over a third-party model is the moment right before you send the prompt. Once the bytes leave your network, you are trusting the provider's controls. Securing the prompt before that hop is the cheapest, highest-impact thing you can do.

This guide focuses on the practical patterns — what to do in code, in which order, and where to put it.

Step 1 — Stop hardcoding provider keys in clients

If your provider key sits in a frontend bundle or in an environment variable that every microservice reads, you have lost the audit trail before you start. Move the key to a single trusted server, or better, to a gateway that holds it for you.

With Privian you can either use the gateway's own pool of provider keys, or bring your own. See BYOK for how the gateway encrypts your provider key at rest.

Step 2 — Route every prompt through one place

Every "what data did we send to OpenAI?" question is much easier to answer when there is one path. A gateway gives you that single chokepoint without forcing every team to share a library.

curl https://api.privian.io/v1/gateway \
  -H "Authorization: Bearer $PRIVIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.5",
    "prompt": "Summarize the ticket from jane@example.com",
    "stream": false
  }'

Step 3 — Mask before you forward

Detection and masking should happen between "I have a prompt" and "I call the provider". With Privian this is the default — the gateway masks recognized entities into deterministic placeholders like EMAIL_1 and forwards the masked text. The mapping lives in memory for the duration of the request and is discarded after the response is rehydrated.

See PII Masking for the full list of supported entity types in the current beta.

Step 4 — Validate and cap the input

Reject prompts that are larger than your policy allows. Privian enforces a hard 32 KiB request body cap by default. Apply your own application-level limits on top — for example, refuse to forward attachments unless they have been explicitly summarized first.

Step 5 — Rehydrate on the way back

The response from the provider will reference the masked tokens. Privian rehydrates those tokens back to the original values before returning the response to your application. The mapping never leaves the gateway. See Rehydration.

Step 6 — Audit, don't archive

You almost certainly want a record that a prompt happened and that masking was applied. You almost certainly do not want raw prompt bodies in your warehouse. Privian's observability records request metadata without storing raw prompts or responses — see Zero retention.

What this does not solve

These steps reduce data exposure. They do not stop a model from following adversarial instructions hidden inside untrusted input. For that, see Prompt security vs. prompt injection.