How a request is handled
What happens in the fraction of a second between an app sending a message and getting an answer. You can see each of these steps for any request: open it in Requests and look at the Decision trace.
- Identify
The API key tells the gateway which app is calling, which team owns it and which governance profile applies. Unknown, revoked or expired keys are refused.
- Classify
Every message is scanned for health data, personal details, card numbers, secrets, government-controlled and confidential data, using patterns, checksums and clinical context. The app can also declare data with the X-AIGov-Sensitivity header.
- Protect
The profile decides what happens to each kind of data: allow it, replace it with placeholders, or block the request. Prompt-injection attempts, denied topics and oversized prompts are stopped here, and a safety model can screen the request.
- Route
Policy rules are checked top to bottom; the first match decides which models may answer. The key's frameworks narrow that further, for example to providers with a BAA. If nothing allows the request, it's refused.
- Call the model
The request goes to the chosen model, with automatic fallback to the next allowed model if one fails. The gateway checks the answer came from the model it chose, and fails closed if not.
- Check the answer
Responses can be scanned for sensitive data and screened by the safety model. Unsafe answers are withheld.
- Record
The decision, data classes, model, tokens, cost and timing are written to a hash-chained audit log. Prompts are stored only if the profile asks for it, encrypted with your organisation's own key.
An example
A clinical note app using the HIPAA · Clinical profile asks for gpt-4o to summarise "Patient John Carter, MRN 00482913, chest pain…".
- The gateway finds health data, so the rule
health-data-on-prem-firstapplies. - HIPAA only allows on-prem models or providers with a signed BAA, so the request is redirected to your on-prem Llama model.
- Because it stays in your data centre, the record number isn't redacted.
- The answer comes back as normal. The response headers say what happened:
X-AIGov-Decision: redirectandX-AIGov-Model: llama-3.1-70b.
Decisions
| Decision | What happened |
|---|---|
allow | Sent to the model the app asked for. |
redirect | Policy didn't allow the requested model for this data, so an allowed one answered. X-AIGov-Model names it. |
deny | Refused with HTTP 403. The error message names the reason and the request ID. |
held | Waiting for a person to approve it under Approvals (HTTP 403, held_for_review). |