Claudey

Coding agents are only as resilient as their API gateways.

Claudey is a local reverse proxy and LLM gateway that lets Claude Code, OpenAI Codex CLI, and Pi run against 37 model providers, from Gemini and DeepSeek to local Ollama.

It's for developers who run coding agents all day, and it's shipped.

It sits between the agent CLI and the inference layer and exposes Anthropic Messages and OpenAI Responses endpoints on localhost.

It translates between them in real time, routes each request by model tier, and fails over along fallback chains when a provider is down. The agent binaries don't change.

Claude Code reaches it through its documented ANTHROPIC_BASE_URL override.¹ Anthropic does not support routing Claude Code to non-Claude models through a gateway, so this is an independent setup, not an endorsed one.

I built Claudey on my own, in Python with FastAPI. My rule: if a proxy can absorb an outage, the agent should never see it. Most of the decisions below follow from that.

Put routing and retries in the proxy, not the agent.

Anything that can go wrong between the agent and a provider should be handled by a local daemon, because the agent can't. Claudey intercepts requests and manages credentials, schema translation, failover routing, and model discovery without modifying the agent.

Long agent loops make this matter.

A run can fire dozens of tool calls within minutes, so one HTTP 429 rate limit in the middle of a refactor can leave the working tree half-edited.

I'd rather the proxy take that hit.

Cost is the second reason. Unit tests, git status checks, and syntax checks cost the same per token as architecture work when they all go through a flagship model.

Claudey's routing is tiered for that reason, and the tiering is optional.

If you want it, you map each agent tier (MODEL_FABLE, MODEL_OPUS, MODEL_SONNET, MODEL_HAIKU) to its own backend and write that mapping yourself. Without one, a single default model applies.

GLOBAL_FALLBACK_MODEL isn't a tier. It's an optional final hop that gets appended to the end of every chain.

Agent asks for

FableOpusSonnetHaiku

• FableGPT AstraNone
• OpusMuse WonderNone
• SonnetGLM 5.3None
• HaikuGemini 4 ArgonNone

FallbackGLM 5.3

Sonnet requests go to GLM 5.3, the fallback.

The third reason is local models.

Ollama, llama.cpp, and LM Studio sit behind the same proxy as the cloud providers, so a security-sensitive project can keep prompts on its own machine.

Wiring those into an agent CLI used to mean monkey-patching binaries or reverse-engineering the client's network layer.

My test for any feature: would the agent have to know about it? If it would, I look for a way to move it into the proxy.

Translate the protocols instead of patching the agents.

Claudey converts every provider's dialect into one internal representation, so each agent sees the Anthropic or OpenAI shape it already expects.

Providers disagree about tool definitions, streaming chunk structures, and role payloads. Claudey streams responses, preserves tool-use JSON schemas, keeps multimodal image inputs, and translates extended thinking blocks.

Tool calls are the sharpest edge

Anthropic represents a tool call as a content block with tool_use and tool_result IDs. OpenAI Chat Completions uses a tool_calls array with stringified arguments, and the /v1/responses endpoint uses function_call items instead.

A streaming parser finds tool execution boundaries on the fly and turns the JSON argument strings into structured blocks.

My ID rule is simple: keep the provider's tool call ID when one exists, and generate one only when it's missing.

# Simplified illustration of the schema normalization
def normalize_tool_call_to_anthropic(openai_tool_call: dict) -> dict:
    return {
        "type": "tool_use",
        "id": openai_tool_call.get("id") or f"call_{uuid.uuid4().hex[:8]}",
        "name": openai_tool_call["function"]["name"],
        "input": json.loads(openai_tool_call["function"]["arguments"]),
    }

The model catalog builds itself

Agent CLIs discover models at startup or when you run /model, and I didn't want a hardcoded list of provider slugs behind that.

Connect a new provider (an NVIDIA NIM key, say, or a local Ollama instance) and Claudey queries its model endpoint, filters for tool-compatible architectures, and puts the results in the agent's native picker.

Models that fail the filter don't show up.

Run fallback chains early, and only while it's safe.

A fallback chain should switch providers only before the first byte reaches the client, and only for failures that a different provider could fix.

A tier can define an ordered chain of targets.

If the primary returns a retryable failure (HTTP 429, a 5xx response, or a socket timeout) before any bytes have been flushed to the client, Claudey rolls the request over to the next node immediately.

Why before the first byte? Because once the client has seen a response begin, you can't un-send it. After that point I let the failure surface. A visible error beats a half-answer quietly stitched to a different model's answer.

A bad API key, a missing permission, or an invalid request won't improve on a second provider, so Claudey doesn't fail over on those either.

My test: could a retry actually change the outcome? If not, the error goes straight to the terminal.

# Multi-tier fallback chain example
MODEL_OPUS="nvidia_nim/nvidia/nemotron-3-super-120b-a12b,open_router/anthropic/claude-sonnet-4-5,zai/glm-5.2"
MODEL_SONNET="gemini/models/gemini-3.1-flash-lite,deepseek/deepseek-chat"

To avoid blind retries, Claudey tracks provider health in memory.

A node that exhausts its quota or goes down enters a temporary cooldown, and later turns skip it instead of paying for another failed attempt.

That state is per process, so a restart wipes it.

Say which model answered, and why

Silent failover would leave you guessing who wrote the answer. Claudey adds an x-claudey-route header to routed Messages and Responses replies, on by default and switchable off. It also emits a structured claudey.api.route.resolved trace event.

HTTP/1.1 200 OK
Content-Type: text/event-stream
x-claudey-route: routeway/meta-llama/llama-3.1-70b; why=down

The text after the semicolon only appears when no model in the chain was usable. It names the health state of the configured primary, here down.

Failovers that happen mid-request show up in the claudey.api.route.failover trace events instead, because the header is built before streaming starts.

A menu bar and system tray companion for macOS and Windows shows health and opens that Admin UI in one click. Optional Discord and Telegram adapters let you send a coding prompt or voice note from your phone.

What Claudey can't do.

• It isn't endorsed. Anthropic doesn't support this setup, so don't expect its help if it breaks.
• It can't make a cheaper model smarter. A fast flash model at the end of a chain is still a fast flash model.
• It can't fail over once a response has started. The error reaches your terminal as it is.
• It can't retry its way out of a bad key. Auth, permission, and invalid-request errors surface straight away.
• Its memory is short. Provider health lives in memory, per process.
• It can still meet a quirk it hasn't seen. A new provider can trip the normalizer.

Does Anthropic support this? No. Claude Code's base URL override is documented, but routing it to non-Claude models is outside what Anthropic supports.

Can it use local models? Yes. LM Studio, llama.cpp, and Ollama are supported alongside the cloud providers, including NVIDIA NIM, Google AI Studio, Vertex AI, Anthropic, DeepSeek, Mistral, Groq, Cerebras, and OpenRouter.

What if a provider fails mid-answer? You see the failure. Claudey only fails over before content starts streaming.

Claudey has not been formally benchmarked, so I have no latency, throughput, or savings numbers and won't quote any.

What I can describe is behavior: tool calls survive translation (with a fallback ID when a provider omits one), and routine work can run on fast models while frontier models stay reserved for hard refactors.

Real savings depend on your tier mapping and workload.

Next time an agent run dies on a 429, ask which layer should have handled it. It's the same restraint I write about in On taste: handle the outage so the agent never has to see it.