[BUG] auth_denied event with reason invalid_token when client posts to App Gateway /v1/metrics endpoint

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

Client version: 2.1.207
Gateway versionL 2.1.207

Metrics flow: Client -> Gateway (containerized) -> ADOT (containerized) -> Amazon Managed Prometheus -> Grafana

Problem: When checking Grafana I don't see any metrics from the EAP versions of the gateway in Grafana when using the explore functionality.

When checking the ADOT container logs and gateway container logs, it appears the gateway is capable of sending some metrics to prometheus as I can see in the ADOT logs relevant user/login information.

When checking the Gateway logs I see the following error repeated several times:
{"ts":"2026-07-13T12:37:58.056Z","evt":"auth.denied","request_id":"7f92434d-f223-4cd2-bc24-083cb2cd27ba","reason":"invalid_token","path":"/v1/metrics","client_ip":"10.242.208.43"}

It is unclear if this is due to some old sessions or if this is a real issue. I'm assuming it is a real issue because I cannot see the metrics in Prometheus or Grafana.

Users are able to log in successfully and use Claude with the apps gateway without any issues. The only thing broken is sending metrics.

What Should Happen?

I am expecting metrics to be posted to the uri successfully and to be able to view them in Prometheus and Grafana. If it is unsuccessful, I am expecting more information in the error message to understand why it doesn't work.

Error Messages/Logs

{"ts":"2026-07-13T12:37:58.056Z","evt":"auth.denied","request_id":"7f92434d-f223-4cd2-bc24-083cb2cd27ba","reason":"invalid_token","path":"/v1/metrics","client_ip":"10.242.208.43"}

Steps to Reproduce

ADOT Config:

# AWS Distro for OpenTelemetry — framework config.
# Receives OTLP from the gateway (telemetry.forward_to) and remote-writes
# metrics to AWS Managed Prometheus using SigV4 against the `aps` service.

extensions:
  sigv4auth:
    region: ${env:AWS_REGION}
    service: aps
    # Uses the default AWS credential chain (instance role / IRSA / env).
    # For ADOT on EC2/ECS/EKS, prefer an instance role over static keys.

  health_check:
    endpoint: 0.0.0.0:13133

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 127.0.0.1:4317
      http:
        endpoint: 127.0.0.1:4318

processors:
  # AMP rejects samples > 1h old and dedupes on (labels, timestamp). Batch
  # tightly enough to stay inside the freshness window.
  batch/metrics:
    timeout: 30s
    send_batch_size: 8192

  resource:
    attributes:
      - key: service.namespace
        value: claude-gateway
        action: upsert
      - key: deployment.environment
        value: ${env:DEPLOY_ENV}
        action: upsert

exporters:
  prometheusremotewrite:
    endpoint: ${env:AMP_REMOTE_WRITE_URL}
    auth:
      authenticator: sigv4auth
    resource_to_telemetry_conversion:
      enabled: true
    # AMP enforces remote-write v1; do not enable send_metadata or v2 features.

  # Uncomment for collector-side debugging only — never leave on in prod.
  debug:
    verbosity: detailed

service:
  extensions: [sigv4auth, health_check]
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [resource, batch/metrics]
      exporters: [prometheusremotewrite, debug]
  telemetry:
    logs:
      level: debug

Gateway Config:

  # ── Listener ─────────────────────────────────────────────────────────────────
  listen:
    host: 0.0.0.0
    port: 8080
    public_url: https://${OIDC_CLIENT_ID}
    # Required behind any TLS-terminating proxy. Used for the IdP redirect_uri
    # and the discovery doc. The gateway does not trust X-Forwarded-* (spoofable).
    # Or terminate TLS in the gateway itself:
    # tls:
    #   cert: /certs/gateway.crt
    #   key:  /certs/gateway.key

  # ── Identity provider ────────────────────────────────────────────────────────
  oidc:
    issuer: ${OIDC_ISSUER}
    client_id: ${OIDC_CLIENT_ID}
    client_secret: ${OIDC_CLIENT_SECRET}
    allowed_groups: [g03094039,g03094040,g03094041]
    groups_claim: ${OIDC_GROUPS_CLAIM}
    userinfo_fallback: true
    scopes: [openid, profile, email]

  # ── Sessions ─────────────────────────────────────────────────────────────────
  session:
    # ≥ 32 bytes of entropy (openssl rand -base64 32). Signs gateway bearer tokens.
    # Single value or array for rotation: index 0 signs, all verify.
    jwt_secret: ${GATEWAY_JWT_SECRET}
    # jwt_secret:
    #   - ${GATEWAY_JWT_SECRET_NEW}    # active signing key
    #   - ${GATEWAY_JWT_SECRET_OLD}    # accepted until outstanding tokens expire
    ttl_hours: 1                       # also bounds revocation latency on IdP deprovision

  # ── Store ────────────────────────────────────────────────────────────────────
  store:
    postgres_url: ${GATEWAY_POSTGRES_URL}       # add ?sslmode=require for managed Postgres

  # ── Upstreams (ordered: failover top→bottom on 5xx/429/timeout/501) ──────────
  upstreams:
    - name: bedrock-in-account
      provider: bedrock
      region: us-east-1
      auth: {}                                  # AWS default credential chain (IRSA / instance role / env)

  auto_include_builtin_models: true

  # ── RBAC + managed settings ─────────────────────────────────────────────────
  # First-match-wins, evaluated top→bottom against the JWT's groups/email claims.
  managed:
    policies:
      - match: { groups: [g-tier-3] }
        settings:
          availableModels: [haiku, sonnet, opus]
      - match: { groups: [g-tier-2] }
        settings:
          availableModels: [haiku, sonnet]
      - match: { groups: [g-tier-1] }
        settings:
          availableModels: [haiku]
      - match: {}
        settings:
          env:
            OTEL_EXPORTER_OTLP_PROTOCOL: http/json
            OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE: cumulative
            CLAUDE_CODE_ENABLE_AUTO_MODE: 1


  # ── Admin ────────────────────────────────────────────────────────────────────
  # admin:
  #   read_key: ${ADMIN_READ_KEY}               # read-only admin (auditors, dashboards)
  #   write_key: ${ADMIN_WRITE_KEY}             # read + write admin
  #   admin_groups: [platform-admins]           # humans administer with their own JWT

  # ── Rate limits ──────────────────────────────────────────────────────────────
  # limits:
  #   rpm: 120                                  # requests per minute, per developer
  # enforcement:
  #   fail_closed_on_error: false               # fail open if counter store unreachable

  telemetry:
    forward_to:
      - url: http://127.0.0.1:4318
        metrics: true
        logs: false
        traces: false

Claude Model

Sonnet (default)

Is this a regression?

Yes, this worked in a previous version

Last Working Version

2.1.175-gateway

Claude Code Version

2.1.207

Platform

AWS Bedrock

Operating System

macOS

Terminal/Shell

Other

Additional Information

_No response_