Skip to main content
Goal: one endpoint on your tailnet that speaks the OpenAI API and fans out to every model provider behind it. After this, choosing a model is a line of config, not a code change.
Principle: Rent the mind. Own the house. Everything that calls a model calls the gateway. The day the best model changes, you change a route and nothing downstream notices. See PRINCIPLES.md.
We use LiteLLM as the gateway, in Docker.

2.1 Install Docker

On the box:

2.2 Get your provider API keys

You need at least one. Start with two or three. Put them in .env (copy from .env.example) — never commit .env. Full provider list LiteLLM supports: https://docs.litellm.ai/docs/providers

2.3 Configure the gateway

The config lives in gateway/litellm.config.yaml. It maps friendly model names to provider/model targets and pulls keys from the environment. It already has Anthropic, Groq, and OpenAI entries — trim to what you have.

2.4 Run it — bound to the tailnet, not the world

gateway/docker-compose.yml binds the proxy to your Tailscale IP on port 4000. Set TAILSCALE_IP and LITELLM_MASTER_KEY in .env first. Then:
The compose file publishes the port as ${TAILSCALE_IP}:4000:4000 — never 0.0.0.0 (see the Docker gotcha in Step 1). LITELLM_MASTER_KEY is the key your tools present to the gateway; make it long and random.

✅ Verification (must pass before Step 3)

From the box or your laptop (on the tailnet), list the models the gateway is serving:
You should get JSON listing claude, fast, gpt (whatever you configured). A quick chat round-trip:
A real reply means the gateway is live and your keys work. From a machine not on the tailnet, the same curl should hang or refuse — that’s the point. This was the single most valuable hour of the build: model choice is now a setting.