Providers & models
A provider is where models run. Add yours under Providers & models; their models are imported and kept up to date automatically.
Add a provider
- Choose the type
In Providers & models (opens in a new tab), click Add provider and pick from about 35 types. Each comes with its default address, the country it processes in and a link to create an API key.
- Paste its API key
Keys are sealed with your organisation's own data key. A provider without a required key shows Needs API key and is never routed to.
- Say where it runs and what you've signed
Mark it on-prem, private cloud or public, and tick the contracts and attestations you actually hold (BAA, DPA, SOC 2, ISO 27001, zero retention…). Frameworks read these to decide where sensitive data may go.
| Kind | Providers |
|---|---|
| Model vendors | OpenAI, Anthropic, Gemini, xAI, Mistral, Cohere, DeepSeek, Moonshot, Qwen, Z.ai, Perplexity |
| Aggregators & inference clouds | OpenRouter, Together, Fireworks, Groq, Cerebras, DeepInfra, SambaNova, Hyperbolic, Novita, Hugging Face, NVIDIA |
| Your cloud tenancy | Azure OpenAI, AWS Bedrock, Vertex AI |
| Self-hosted | Ollama, vLLM, NVIDIA NIM, TGI, LM Studio, LiteLLM, any OpenAI-compatible endpoint |
The full list is on agentkeyroute.com/models.
Automatic model sync
- Runs when a provider is added, when its key changes, every 6 hours, and when you click Sync models.
- Reads the provider's own model list with its key; prices and context windows come from a public model catalogue.
- Each model is typed (chat, embedding, image, speech, rerank, moderation, video) and guard models are detected. Each endpoint only routes to models of its type.
- Models the provider stops listing are disabled, not deleted.
Reliability
- If a model fails, the request falls back to the next model the rules allow. Providers that stop answering are taken out of rotation until they recover.
- Per provider, set the maximum concurrent requests and rate-limit protection. The provider page shows success rate, latency and queue.
- The gateway checks which model actually answered and fails closed (
served_model_mismatch) if it wasn't one policy allowed.
Response cache
Turned on per governance profile. Entries are encrypted and kept separate for each organisation, profile and model chain; the cache key is the request after redaction. Health, card and secret data are never cached by default. X-AIGov-Cache says hit or miss, and Cost & usage shows the savings.
OpenRouter and zero data retention
OpenRouter sends each request to one of several companies. The gateway governs that choice:
- Sensitive data only goes to zero-retention endpoints. The gateway names them explicitly, so endpoints that keep flagged prompts are excluded whatever your OpenRouter account settings. Per provider, choose When data is sensitive (default), Always or Off.
- Zero-retention marking. Models with such routes are marked
ZDRandNO_TRAINING, refreshed hourly, so rules and frameworks can use them. - Where it actually ran. After each call the request records which company served it, in which region, at what cost and after which fallbacks.
- Exact cost. OpenRouter's reported cost replaces the price-list estimate.