Gateway: per-model `upstream_model` routing is not honored (documented "omit an upstream to skip it" lever is inert)

Status Open
Reported on v2.1.220
Maintainer reply None cached
Activity 0 comments · opened Aug 6, 2026

The configuration reference documents model-scoped routing: "Per-upstream model IDs are keyed on the upstream's name:; an upstream without a name: defaults to its provider string (e.g. bedrock). Any upstream not listed for a model is skipped." We cannot get this to work on gateway 2.1.220 or 2.1.222.

Config (sanitized):

upstreams:
  - provider: bedrock
    region: us-east-1
    auth: { aws_bearer_token: ${...} }
  - provider: anthropic
    auth: { api_key: ${...} }

auto_include_builtin_models: false
models:
  - id: claude-fable-5
    label: Claude Fable 5
    upstream_model:
      anthropic: claude-fable-5     # bedrock omitted -> should be skipped
  # (opus/sonnet/haiku entries with both bedrock + anthropic keys)

Expected: claude-fable-5 requests skip bedrock ("without a network round-trip") and are served by anthropic.

Actual: every claude-fable-5 request was forwarded to bedrock, which (before we fixed an unrelated AWS account setting) returned 400 data retention mode 'default' is not available for this model — and since 400 is not failover-eligible, the request died there rather than reaching the anthropic upstream that the map pinned it to.

Evidence:

  • Identical behavior with the models: block present and absent, and with auto_include_builtin_models true and false — the block appears to have no routing effect at all.
  • Reproduced on gateway binary 2.1.220 and, after upgrading specifically to test this, 2.1.222.
  • The boot log names every other config section it loads (oidc, upstreams, telemetry, managed settings) but never mentions the model catalog.
  • Translation does work for models listed with a bedrock key (e.g. claude-sonnet-5us.anthropic.claude-sonnet-5 served fine), so either the builtin catalog or mechanical translation is active — only the skip semantics never engage.

Config validates at boot (no unknown-key errors). If the lever has a minimum version above 2.1.222 or an undocumented prerequisite, a version gate in the docs (like the one on 404-failover, 2.1.198+) would prevent others burning an evening on this the way we did. Related context: #82991 (failover status set), filed by the same org.

View original on GitHub ↗