Glossary
What is an LLM gateway?
An LLM gateway is a routing layer that accepts AI requests from an application and forwards them to one or more model providers.
Definition
An LLM gateway is a routing layer that accepts AI requests from an application and forwards them to one or more model providers.
Definition
LLM gateway — short definition
Why it matters
Why this matters
Without a gateway, every client calls providers directly. There is no central place to mask sensitive data, apply policy, rotate keys or observe usage. A gateway gives security and platform teams one enforcement point for all AI traffic.
Without a gateway, every client calls providers directly. There is no central place to mask sensitive data, apply policy, rotate keys or observe usage. A gateway gives security and platform teams one enforcement point for all AI traffic.
How it works
How it works
Framework
LLM gateway process
- 01
Accept
The gateway accepts HTTPS/JSON requests from your app (prompt + model).
- 02
Resolve
It picks the provider and model based on the request and your configuration.
- 03
Forward
It calls the provider with your credentials and returns the response.
Implementation
Learn how this works in Privian
From definition to implementation, docs and architecture — the same idea at different layers.
FAQ
Frequently asked questions
- What is an LLM gateway used for?
- Centralizing routing, authentication, model selection, observability and — in a privacy-first gateway — masking and prompt security for all AI traffic.
- Is an LLM gateway the same as an AI gateway?
- The terms overlap. 'AI gateway' is often used more broadly; 'LLM gateway' specifically denotes a routing layer for large-language-model traffic.
- Do I need an LLM gateway if I only use one provider?
- Yes if you want a single enforcement point for masking, prompt security, observability and key rotation. No if you only need basic routing.
Where to go next
From definition to implementation
This entry is part of the Privian content library. If you are evaluating how this concept is implemented in practice, the links below are the most direct paths.
Related product
Privian LLM Gateway
A privacy-first LLM gateway with PII masking, BYOK and rehydration applied at the edge.
Open product page →