output_config.format returns 400 'compiled grammar is too large' with no documented limits

Resolved 💬 2 comments Opened May 2, 2026 by herrhaase Closed May 5, 2026

Description

When using output_config.format with a JSON schema that has more than approximately 24 optional parameters, the API returns:

400: "compiled grammar is too large"

This limit was discovered empirically through trial and error. There is no documentation on:

  • What the compiled grammar size limit is
  • How schema features (optional fields, enums, nested objects, arrays with maxItems) contribute to grammar size
  • How to estimate whether a given schema will compile successfully before sending a request

Schema features that increase grammar size (discovered empirically)

  • Optional fields (each roughly doubles the FSM state space)
  • Large enums (100+ values)
  • maxItems on arrays (grammar must track count)
  • Deeply nested objects
  • description fields (unclear if they affect compilation, but they increase raw schema size)

Expected behavior

Documentation should specify grammar size limits and how schema features contribute to compiled size. Ideally, the error message should indicate which part of the schema is causing the limit to be exceeded, or provide an estimated vs. maximum grammar size.

Workaround

Make all fields required (eliminates the optional-field state space doubling). Strip description fields from schemas before passing to the API. Avoid large enums and maxItems. Flatten nested objects. If the schema still exceeds the limit, fall back to unconstrained text generation with manual JSON parsing.

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗