Gateway: per-model `upstream_model` routing is not honored (documented "omit an upstream to skip it" lever is inert)
The configuration reference documents model-scoped routing: "Per-upstream model IDs are keyed on the upstream's name:; an upstream without a name: defaults to its provider string (e.g. bedrock). Any upstream not listed for a model is skipped." We cannot get this to work on gateway 2.1.220 or 2.1.222.
Config (sanitized):
upstreams:
- provider: bedrock
region: us-east-1
auth: { aws_bearer_token: ${...} }
- provider: anthropic
auth: { api_key: ${...} }
auto_include_builtin_models: false
models:
- id: claude-fable-5
label: Claude Fable 5
upstream_model:
anthropic: claude-fable-5 # bedrock omitted -> should be skipped
# (opus/sonnet/haiku entries with both bedrock + anthropic keys)
Expected: claude-fable-5 requests skip bedrock ("without a network round-trip") and are served by anthropic.
Actual: every claude-fable-5 request was forwarded to bedrock, which (before we fixed an unrelated AWS account setting) returned 400 data retention mode 'default' is not available for this model — and since 400 is not failover-eligible, the request died there rather than reaching the anthropic upstream that the map pinned it to.
Evidence:
- Identical behavior with the
models:block present and absent, and withauto_include_builtin_modelstrue and false — the block appears to have no routing effect at all. - Reproduced on gateway binary 2.1.220 and, after upgrading specifically to test this, 2.1.222.
- The boot log names every other config section it loads (
oidc,upstreams,telemetry,managed settings) but never mentions the model catalog. - Translation does work for models listed with a bedrock key (e.g.
claude-sonnet-5→us.anthropic.claude-sonnet-5served fine), so either the builtin catalog or mechanical translation is active — only the skip semantics never engage.
Config validates at boot (no unknown-key errors). If the lever has a minimum version above 2.1.222 or an undocumented prerequisite, a version gate in the docs (like the one on 404-failover, 2.1.198+) would prevent others burning an evening on this the way we did. Related context: #82991 (failover status set), filed by the same org.