Smart routing

When the model a request names is down, smart routing answers with a close alternative instead of an error. Your app keeps working through an outage without a line of retry code.

How it works

1. Every request first goes to the model you asked for. Most problems end here: if one path to the model stalls or errors, the request is retried on another before you see anything.
2. Only when the model itself can’t answer (every path failed, or none is available) does smart routing step in.
3. It tries the alternatives in order, each with the same self-healing, until one answers.
4. The answer tells you which model replied (model in the body, x-zurelay-model and x-zurelay-routed-from headers), and it’s billed at that model’s price.

When it’s on, the model you asked for gets about half of the usual time before the alternatives take over, so an outage costs seconds, not minutes. A model known to be down is skipped straight away.

Turning it on

A key’s setting is one of:

Per request

A list of alternatives

Pass models next to model: up to three alternatives, tried in order if model is down, whatever the key’s setting or the header. It’s never sent to the model. The OpenAI Python SDK sends it through extra_body.

import os
from openai import OpenAI
client = OpenAI(
    base_url="https://api.zurelay.com/v1",
    api_key=os.environ["ZURELAY_API_KEY"],
)
response = client.chat.completions.create(
    model="gpt-6-sol",
    messages=[
      {
        "role": "user",
        "content": "Summarize this ticket in one line: ..."
      }
    ],
    extra_body={
      "models": [
        "gpt-6.1-sol",
        "gpt-6-astra"
      ]
    },
)
print(response.choices[0].message.content)

The header

x-zurelay-fallback: off    # never reroute this request
x-zurelay-fallback: auto   # use the automatic alternatives for this request

What applies, first to last: the request’s models, then the header, then the key’s setting, then the workspace default.

What alternatives can answer

• They read what you sent. A request with a PDF or a video only goes to alternatives that read it. If none does, you get the error rather than an answer that ignores your file.
• The key may use them. Alternatives outside a key’s allowed models are skipped.
• They speak the endpoint. On /v1/messages, alternatives are Claude models.
• The request itself is fine. If the model refused the request (a bad parameter, too long a prompt), it isn’t rerouted: the alternatives would refuse it too.

Automatic alternatives

Live from the catalog: the alternatives each model falls back to in automatic mode.

Things to know

• An alternative may cost more or less than the model you asked for. Your request log shows the model that answered and its cost.
• Streaming works the same: nothing is sent until a model is answering, and the chunks name that model.
• Images and videos aren’t rerouted: a different image or video model makes a different picture.