[BUG] /goal evaluator schema rejected by strict backends: required omits impossible

Status Open
Reported on v2.1.274
Maintainer reply None cached
Activity 1 comment · opened Sep 17, 2026

What's Wrong?

The /goal Stop-hook evaluator sends an output_config.format JSON schema whose required array omits one of its own properties (impossible). Any backend that enforces the strict schema subset (required must list every key in properties) rejects every evaluation with HTTP 400. Making this schema strict-subset compliant is free hardening: it changes nothing against Anthropic's own API and unbreaks every strict backend at once.

Reproduced against Meta Model API /v1/messages (Anthropic-compatible endpoint, model muse-spark-1.3): each turn that reaches Stop while a goal is active fails and the goal never evaluates.

The schema, from the 2.1.274 binary:

{"type":"object","properties":{"ok":{"type":"boolean"},"reason":{"type":"string"},"impossible":{"type":"boolean"}},"required":["ok","reason"],"additionalProperties":false}

What Should Happen?

The evaluator schema should be strict-compatible: required lists every key in properties (["ok","reason","impossible"]). In the 2.1.274 binary the verdict gate reads as a strict === true check on impossible, so requiring the key should change no verdict logic.

Error Messages/Logs

Stop hook error: Hook evaluator API error: API Error: 400 'required' is required to be supplied and to be an array including every key in properties. Missing 'impossible'.

Steps to Reproduce

Session-level:

  1. Route Claude Code to Meta Model API (ANTHROPIC_BASE_URL=https://api.meta.ai, model muse-spark-1.3). Any strict-validating Anthropic-compatible backend applies.
  2. Run /goal with any condition.
  3. End a turn so it reaches Stop.
  4. Observe the Stop hook error above. The goal never evaluates.

Backend-level (proves the schema, not the session, is at fault; key redacted):

# 400, exact message from the report above
curl https://api.meta.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $META_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"muse-spark-1.3","max_tokens":50,"messages":[{"role":"user","content":"Reply with exactly: probe-ok"}],"output_config":{"effort":"max","format":{"type":"json_schema","schema":{"type":"object","properties":{"ok":{"type":"boolean"},"reason":{"type":"string"},"impossible":{"type":"boolean"}},"required":["ok","reason"],"additionalProperties":false}}}}'
# 200: identical request with "required":["ok","reason","impossible"]
curl https://api.meta.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $META_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"muse-spark-1.3","max_tokens":50,"messages":[{"role":"user","content":"Reply with exactly: probe-ok"}],"output_config":{"effort":"max","format":{"type":"json_schema","schema":{"type":"object","properties":{"ok":{"type":"boolean"},"reason":{"type":"string"},"impossible":{"type":"boolean"}},"required":["ok","reason","impossible"],"additionalProperties":false}}}}'

A request with no output_config also returns 200, so auth, shape, and model are fine.

Environment

  • Claude Model: Other (muse-spark-1.3 via third-party backend)
  • Regression: I don't know
  • Claude Code Version: 2.1.274 (updated from 2.1.273 today and re-verified the schema in the new binary before filing)
  • Platform: Other (Meta Model API, Anthropic-compatible)
  • Operating System: macOS
  • Terminal: Terminal.app (macOS)

Additional Information

  • Searched open issues for goal evaluator impossible required: no duplicate found. Related but different cause: #80149 (400 on deferred MCP tool refs). Precedent for the fix direction: #37746 (closed completed, maintainer: fixed in next release; #54224 closed as its duplicate). That case was the same class with a different trigger: Vertex AI rejected the whole output_config param, while here a strict backend rejects a strict-subset violation. Same fix direction: client-side.
  • Suggested fix: add "impossible" to required in the hook evaluator output schema.
  • Preflight: searched existing issues, single bug, latest version.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗