Unified endpoint
One base URL, one key and one request shape in front of Azure OpenAI and other OpenAI-compatible providers. Adding an upstream becomes a configuration change instead of a release.
Generally availableAI API gateway
One endpoint in front of every model you call.
RouteMind sits between your application and the providers that serve your models. You integrate once. We resolve the route, move to another upstream when one degrades, answer repeat calls from cache, and keep the usage record your finance team keeps asking for.
Drop-in for the OpenAI-compatible base URL you already use.
Request pathroute · cache · retry · meter
Your application sends one request to RouteMind. RouteMind matches it to a route policy, checks the cache, calls the selected upstream provider, moves to the next upstream if that call does not succeed within the retry budget, records the usage, and returns a single response to your application.
What the gateway does
Four jobs, on every request.
Automatic failover
Timeouts, rate limits and 5xx responses move to the next healthy upstream inside the same request. Your client sees one call, not the retry loop you would otherwise write and maintain.
Generally availableRequest caching
Repeat calls are answered from cache instead of the provider, with a TTL you set per route and an explicit bypass for anything that has to be fresh.
Generally availableUsage analytics
Every request leaves a record: model, upstream, latency, token counts, cache outcome and cost attribution, grouped by key, route and application.
Generally availableRequest lifecycle
What happens between your call and the answer.
Nothing here is hidden from you. Each stage is configurable, and each one reports what it did on the response.
- 01
Point at the gateway
Change the base URL in the SDK you already use and issue a RouteMind key. Request and response bodies stay exactly as they are.
- 02
Resolve the route
A route policy maps the model you asked for to an ordered list of upstream deployments, with weights, region pins and per-key overrides.
- 03
Call, retry, fail over
The first healthy upstream serves the call. Retryable failures move down the list inside your latency budget rather than surfacing to your client as an error.
- 04
Record and bill
Tokens, latency, cache outcome and upstream attribution land in the usage record — the same record your subscription and metered charges are drawn from.
Integration
Two lines, not a migration.
RouteMind accepts the OpenAI-compatible request shape your SDK already sends. Swap the base URL and the key; keep your prompts, your streaming, your tool calls and your error handling.
Beforepython
from openai import OpenAI
client = OpenAI(
base_url="https://<resource>.openai.azure.com/openai/v1",
api_key=AZURE_OPENAI_API_KEY,
)Afterpython
from openai import OpenAI
client = OpenAI(
base_url="https://api.routemind.dev/v1",
api_key=ROUTEMIND_API_KEY,
)Failover, caching and usage analytics apply from the first request. No SDK to install, no sidecar to run, no change to your prompt code.
Call optimization
The three things that actually move your bill and your latency.
Failover
Degradation handled inside the request, not in your application code.
- Health signal
- Per-upstream error rate, latency and rate-limit headers, evaluated continuously rather than guessed per request.
- Retry budget
- A latency ceiling you set. Attempts stop at the budget instead of stacking until your client gives up.
- Ordering
- Weighted primaries, then declared fallbacks. Region and residency pins are honoured before weight.
- Visibility
- The upstream that served the call and the number of attempts come back on response headers.
Caching
The cheapest provider call is the one you never make.
- Match
- A normalized request signature: model, messages, tools and the sampling parameters that change the output.
- TTL
- Set per route — short for retrieval-backed answers, longer for stable classification and extraction work.
- Bypass
- A per-request header opts any call out of the cache, and high-temperature requests are excluded by default.
- Effect
- A hit skips the upstream entirely: no provider tokens spent, and the response returns at gateway latency.
Analytics
One usage record, queryable by the people who have to explain it.
- Dimensions
- Model, upstream, route, API key, application and cache outcome.
- Measures
- Requests, prompt and completion tokens, cache hit rate, error rate, p50 and p95 latency.
- Attribution
- Spend rolled up by key and by application, so an internal team or a customer account can be charged back.
- Export
- Usage is queryable through the API and downloadable for the billing period.
Commercial model
Subscription plus metered usage.
The plan pays for the platform. Metering covers what actually flowed through it. Provider charges stay where they already are — on your upstream accounts — and are shown next to gateway usage so both sides reconcile.
- Subscription
- A monthly plan per organization, sized by seats, routes and analytics retention.
- Metered usage
- A routing fee on the tokens that pass through the gateway. Cache hits are not billed as routed tokens.
- Upstream cost
- Billed by your provider on your own account, and surfaced in RouteMind analytics for reconciliation.
- Invoicing
- One monthly invoice combining the plan and the metered line, with the usage record attached.
Get started
Point one base URL at RouteMind.
The quickstart runs about ten minutes end to end: issue a key, declare an upstream, send your first routed request, and read the usage record it produced.