LiteLLM proxy and SDK with DeepSeek, Claude and any OpenAI-compatible API

What LiteLLM does

• The SDK is one Python function, completion(), that calls any provider in OpenAI's format. The prefix on the model name picks the provider: openai/ means any OpenAI-compatible API.
• The proxy is a server you run yourself, on http://localhost:4000 by default. Apps call it in OpenAI's or Anthropic's format, and it adds keys, budgets, logging and fallbacks in front of your providers.

Set up the proxy

1. Install it in a virtual environment. LiteLLM 1.106 needs Python 3.10 or later.

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install "litellm[proxy]"

2. Write config.yaml. This is the file we tested: DeepSeek V4.1 Flash through the OpenAI-compatible endpoint, Claude Sonnet 5.5 through LiteLLM's Anthropic provider, and a wildcard that reaches every other model as zurelay/<model ID>. The model_info prices are Zurelay's current ones.

model_list:
  - model_name: deepseek-v4.1-flash
    litellm_params:
      model: openai/deepseek-v4.1-flash
      api_base: https://api.zurelay.com/v1
      api_key: os.environ/ZURELAY_API_KEY
      use_chat_completions_api: true
    model_info:
      input_cost_per_token: 0.00000003
      output_cost_per_token: 0.00000012
      cache_read_input_token_cost: 0.00000000074
  - model_name: claude-sonnet-5-5
    litellm_params:
      model: anthropic/claude-sonnet-5-5
      api_base: https://api.zurelay.com
      api_key: os.environ/ZURELAY_API_KEY
    model_info:
      input_cost_per_token: 0.00000079
      output_cost_per_token: 0.00000395
      cache_read_input_token_cost: 0.000000079
  - model_name: zurelay/*
    litellm_params:
      model: openai/*
      api_base: https://api.zurelay.com/v1
      api_key: os.environ/ZURELAY_API_KEY
      use_chat_completions_api: true
litellm_settings:
  use_chat_completions_url_for_anthropic_messages: true
  check_provider_endpoint: true
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

3. Start it with your Zurelay key and a master key of your own. Clients send the master key; LiteLLM expects keys that start with sk-.

export ZURELAY_API_KEY="zr_live_..."
export LITELLM_MASTER_KEY="sk-choose-a-long-random-key"
litellm --config config.yaml

4. Call it like any OpenAI-compatible API, with the model_name from the config.

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4.1-flash", "messages": [{"role": "user", "content": "Hello"}]}'

Or use the Python SDK

Without the proxy, pass the base URL and key on each call. For Claude through LiteLLM's anthropic/ provider, the base URL is https://api.zurelay.com, with no /v1.

import os
from litellm import completion
response = completion(
    model="openai/deepseek-v4.1-flash",
    api_base="https://api.zurelay.com/v1",
    api_key=os.environ["ZURELAY_API_KEY"],
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
claude = completion(
    model="anthropic/claude-sonnet-5-5",
    api_base="https://api.zurelay.com",
    api_key=os.environ["ZURELAY_API_KEY"],
    messages=[{"role": "user", "content": "Hello"}],
)

Claude Code through LiteLLM

The proxy also serves Anthropic's Messages API, so Claude Code can run on any model in the config. Point Claude Code at the proxy and name the models it should use:

export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
export ANTHROPIC_MODEL="deepseek-v4.1-flash"
export ANTHROPIC_DEFAULT_OPUS_MODEL="claude-sonnet-5-5"
export ANTHROPIC_DEFAULT_SONNET_MODEL="claude-sonnet-5-5"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="deepseek-v4.1-flash"
claude

For Claude models alone you don't need LiteLLM: Claude Code talks to Zurelay directly, as Claude Code pricing shows. For a router with a web page instead of a YAML file, see Claude Code Router.

The settings that broke it

1. Claude Code failed on DeepSeek with Unknown request URL: POST /v1/responses. For openai/ models, LiteLLM turns Anthropic Messages requests into calls to OpenAI's Responses API, which Zurelay doesn't serve. After one failure LiteLLM also put the model in cooldown, so the next requests got 429 No deployments available for selected model. use_chat_completions_url_for_anthropic_messages: true under litellm_settings sends them to Chat Completions instead.
2. Responses API clients failed the same way. A request to the proxy's /v1/responses, or litellm.responses() in the SDK, went upstream unchanged until use_chat_completions_api: true was set on the model.
3. Claude with /v1 in the base URL called /v1/v1/messages and got a 404. The anthropic/ provider adds /v1/messages itself; openai/ models need the /v1.
4. The wildcard listed the wrong models. With openai/*, the proxy's /v1/models showed LiteLLM's built-in list of 216 OpenAI names (gpt-3.5-turbo, babbage-002) and none of Zurelay's. Requests for real model IDs still worked. check_provider_endpoint: true made it list Zurelay's models instead.
5. On Windows, the proxy crashed at startup with UnicodeEncodeError: 'charmap' codec can't encode characters when its output went to a log file: the banner has characters the default code page lacks. Python's UTF-8 mode fixed it: PYTHONUTF8=1 in the environment, or python -X utf8.

Cost tracking

LiteLLM's spend tracking prices each request from its own list. For models it knows, that's the lab's list price: it counted our Claude Sonnet 5.5 request at Anthropic's rate, about 2.5 times what Zurelay billed. For models it doesn't know, such as DeepSeek V4.1 Flash behind openai/, it records no cost at all. The model_info lines in the config above fix both: with them, the cost LiteLLM returned in its x-litellm-response-cost header matched our bill.

What it cost in our test

Through the proxy, Claude Code wrote a small Python file on DeepSeek V4.1 Flash, then edited it on Claude Sonnet 5.5. Both used Claude Code's file tools correctly. Zurelay billed:

Each request carried Claude Code's instructions and tool definitions, 13,000 to 18,000 tokens, and from the second request on most of that was read from the cache. Claude Code's own estimate for the Claude task was $0.054, at Anthropic's list prices. The SDK calls before it, one short answer each, cost $0.000006 on DeepSeek and $0.0003 on Claude Sonnet 5.5.

Things to know

• The proxy listens on every network interface (0.0.0.0) unless you pass --host 127.0.0.1. Keep the master key set, or anyone on your network can spend your credit.
• Wildcard models take their real ID after the prefix: zurelay/gpt-6.1-terra in a request becomes gpt-6.1-terra at Zurelay.
• On Windows, install into a short folder path. A virtual environment deep in a temp folder failed with ModuleNotFoundError: No module named 'openai.types...' because some file paths passed Windows' 260-character limit.

Prices for the models you'd put behind it

Per 1M tokens, the lab's list price and Zurelay's:

Tested on November 3, 2026 with LiteLLM 1.106.6 on Python 3.12 and Claude Code 2.1.317, on Windows, against Zurelay's API. The short setup is in LiteLLM in the docs. LiteLLM's own documentation: OpenAI-compatible endpoints and proxy config.