[FEATURE] Third-party inference: configurable model-discovery endpoint (hardcoded /v1/models)

Status Open
Maintainer reply None cached
Activity 0 comments · opened Sep 1, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

When Claude Desktop's third-party inference is configured with a gateway provider and modelDiscoveryEnabled, the app fetches the model list from <inferenceGatewayBaseUrl>/v1/models. That suffix is hardcoded.

Not every OpenAI-compatible gateway serves its catalog under /v1 - some expose it at /models (or another path) and return 404 for /v1/models, so discovery fails:

Model discovery — Gateway /v1/models returned HTTP 404.

Proposed Solution

A config key alongside modelDiscoveryEnabled - either:

  • modelDiscoveryPath (resolved against inferenceGatewayBaseUrl), or
  • modelDiscoveryUrl (absolute),

so deployments whose gateway uses a different layout can still use discovery. Available through the same surfaces as the other inference keys (config library, MDM, registry/ADMX).

Alternative Solutions

A hand-maintained inferenceModels list works, but it goes stale as the gateway's catalog evolves and has to be re-edited on every model release - exactly what modelDiscoveryEnabled exists to avoid.

Priority

Medium - Would be very helpful

Feature Category

Configuration and settings

Use Case Example

  1. An admin deploys Claude Desktop against an internal OpenAI-compatible gateway that serves its catalog at /models.
  2. modelDiscoveryEnabled is turned on so the model picker tracks the gateway's catalog.
  3. Discovery 404s; the admin falls back to a static inferenceModels list.
  4. With modelDiscoveryPath: "/models", the picker would track the catalog automatically.

Additional Context

Related issues:

  • #55182 - docs for the /v1/models discovery behavior (closed)
  • #88345 - gateway mode capability discovery / stale model selection
  • #80626 - inferenceModels[].prefer1m regression (an example of the static-list maintenance burden)

View original on GitHub ↗