Get started

How a request is handled

What happens in the fraction of a second between an app sending a message and getting an answer. You can see each of these steps for any request: open it in Requests and look at the Decision trace.

  1. Identify

    The API key tells the gateway which app is calling, which team owns it and which governance profile applies. Unknown, revoked or expired keys are refused.

  2. Classify

    Every message is scanned for health data, personal details, card numbers, secrets, government-controlled and confidential data, using patterns, checksums and clinical context. The app can also declare data with the X-AIGov-Sensitivity header.

  3. Protect

    The profile decides what happens to each kind of data: allow it, replace it with placeholders, or block the request. Prompt-injection attempts, denied topics and oversized prompts are stopped here, and a safety model can screen the request.

  4. Route

    Policy rules are checked top to bottom; the first match decides which models may answer. The key's frameworks narrow that further, for example to providers with a BAA. If nothing allows the request, it's refused.

  5. Call the model

    The request goes to the chosen model, with automatic fallback to the next allowed model if one fails. The gateway checks the answer came from the model it chose, and fails closed if not.

  6. Check the answer

    Responses can be scanned for sensitive data and screened by the safety model. Unsafe answers are withheld.

  7. Record

    The decision, data classes, model, tokens, cost and timing are written to a hash-chained audit log. Prompts are stored only if the profile asks for it, encrypted with your organisation's own key.

An example

A clinical note app using the HIPAA · Clinical profile asks for gpt-4o to summarise "Patient John Carter, MRN 00482913, chest pain…".

  • The gateway finds health data, so the rule health-data-on-prem-first applies.
  • HIPAA only allows on-prem models or providers with a signed BAA, so the request is redirected to your on-prem Llama model.
  • Because it stays in your data centre, the record number isn't redacted.
  • The answer comes back as normal. The response headers say what happened: X-AIGov-Decision: redirect and X-AIGov-Model: llama-3.1-70b.

Decisions

DecisionWhat happened
allowSent to the model the app asked for.
redirectPolicy didn't allow the requested model for this data, so an allowed one answered. X-AIGov-Model names it.
denyRefused with HTTP 403. The error message names the reason and the request ID.
heldWaiting for a person to approve it under Approvals (HTTP 403, held_for_review).