Quickstart

From your existing SDK to a routed request.

RouteMind speaks the OpenAI-compatible request shape your client already sends. The integration is a base URL and a key; everything below is what you get once that swap is in place.

https://api.routemind.dev/v1

Start from what you already have

A typical integration points an OpenAI-compatible client straight at one provider deployment. It works until that deployment is rate limited, degraded or in the wrong region.

Existing clientpython

from openai import OpenAI

client = OpenAI(
    base_url="https://<resource>.openai.azure.com/openai/v1",
    api_key=AZURE_OPENAI_API_KEY,
)

Change the base URL and the key

Swap in the RouteMind base URL and a RouteMind key. Request and response bodies are unchanged, so prompts, streaming and tool calls carry over untouched.

Routed clientpython

from openai import OpenAI

client = OpenAI(
    base_url="https://api.routemind.dev/v1",
    api_key=ROUTEMIND_API_KEY,
)

Declare the upstreams behind the route

A route names the model your application asks for and the deployments allowed to answer it. Weights split steady-state traffic; the failover entry catches the case where the primaries cannot.

route.yamlyaml

# route.yaml — one policy, an ordered list of upstreams
route: gpt-4o-mini
cache:
  ttl: 300s
upstreams:
  - provider: azure-openai
    deployment: gpt-4o-mini-eastus
    weight: 80
  - provider: azure-openai
    deployment: gpt-4o-mini-westeu
    weight: 20
  - provider: openai-compatible
    deployment: fallback-pool
    role: failover

Send the first routed request

Any HTTP client works — the gateway is a plain OpenAI-compatible endpoint. This is the shortest way to confirm the key, the route and the upstream credentials are all in place.

Requestshell

curl https://api.routemind.dev/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEMIND_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Ping"}]
  }'

Read what the gateway did

Every response reports its own routing. Here the first upstream did not answer, the second one did, and the result was not served from cache — visible without opening a dashboard.

Response headershttp

HTTP/1.1 200 OK
x-routemind-provider: azure-openai/gpt-4o-mini-westeu
x-routemind-route: gpt-4o-mini
x-routemind-cache: miss
x-routemind-attempts: 2
x-routemind-request-id: rq_8f3c21d0a4

Response contract

Headers on every routed response.

These are additions. The response body stays byte-compatible with the upstream provider, so nothing in your parsing changes.

Header	Reports
x-routemind-provider	The upstream provider and deployment that served the request.
x-routemind-route	The route policy, and its version, that was applied.
x-routemind-cache	hit, miss or bypass.
x-routemind-attempts	How many upstream attempts were made, including failover.
x-routemind-request-id	The identifier to quote in support and to look up in analytics.

Behaviour

What to expect at the edges.

Errors: Upstream error bodies are preserved and passed through. Gateway errors are distinguishable from provider errors.
Streaming: Server-sent events pass through unchanged. Failover happens before the first token, not mid-stream.
Timeouts: Bounded by the retry budget on the route. When it is exhausted, the last upstream error is returned as-is.
Rate limits: Your per-key limits are enforced at the gateway, before an upstream is called or metered.
Idempotency: A failed-over request is metered once, on the attempt that produced the response.

This quickstart is the integration path, not the full API reference. Reference documentation ships with your account; if something here does not match what you see, tell us and we will fix the page.

Talk to us How routing works