FAQ
Concepts
What's the difference between a router and a gateway?
A gateway handles how requests reach models, and a router decides which model serves each one. Emissary can be both: one gateway URL with a router behind it, which we recommend, or the routing policy alongside a gateway you already run. See routers and gateways.
What's the difference between complexity-based and performance-based routing?
Complexity-based routing guesses how hard a task is from its input and picks a cheap or expensive model, and that's hard to do reliably. Performance-based routing, which is what Emissary does, asks which models can actually do the task, based on how they've performed. See the router design space.
Does the router choose effort, or only the model?
Both. Changing a model's effort keeps you on the same model, so the prompt cache is preserved, while switching models loses it. See what the router sees.
Can the router handle images?
The router knows which models accept images and won't send an image request to a text-only model. It doesn't interpret the image content itself. See what the router sees.
What's the relationship to the older router quickstart?
The earlier router quickstart describes the stateless router. The routers documented in this section are stateful, model-aware, and rank on cache state, output length and effort.
Quality, cost and latency
Does routing reduce quality?
It shouldn't, and it can improve it. Models are strong at different things, so routing each request to the model that handles it best can outperform any single model on your workload as a whole. The router first keeps models it's confident can do the task, and only then ranks them by cost. See how a routing decision is made.
How much will I save?
It depends on your traffic. A workload where most requests need a frontier model saves less than one with many routine requests. Cache and effort decisions add savings for agentic workloads. Measure on a sample of your own traffic. See run a trial.
Does the router add latency?
A routing decision takes some time, so a router adds a small amount to each request. Requests that go to a faster model finish sooner. Measure end-to-end latency on your workload rather than the routing step alone.
Using Emissary
Which models can I route between?
The models in your router's catalog, which you choose when you create it. See choose your models.
Can I create a router through the API?
Not yet. Routers are created and configured in the dashboard. Once created, you call a router through the gateway API. See get started.
How do I know why a request went to a particular model?
Click into the request in Activity. You'll see each model's confidence and the routing reason. See monitor and debug.
What happens when a new model is released?
The router already scores models you haven't enabled, so you can see whether it could take a meaningful share of your traffic. Enabling it needs no application changes. See migrate to new models.
Choosing
When shouldn't I use a router?
When you can select models offline, for example a workflow with fixed steps. See when to route, and when not to.
How do I choose a router?
Use the router selection checklist.