output_config.format returns 400 'compiled grammar is too large' with no documented limits
Description
When using output_config.format with a JSON schema that has more than approximately 24 optional parameters, the API returns:
400: "compiled grammar is too large"
This limit was discovered empirically through trial and error. There is no documentation on:
- What the compiled grammar size limit is
- How schema features (optional fields, enums, nested objects, arrays with
maxItems) contribute to grammar size - How to estimate whether a given schema will compile successfully before sending a request
Schema features that increase grammar size (discovered empirically)
- Optional fields (each roughly doubles the FSM state space)
- Large enums (100+ values)
maxItemson arrays (grammar must track count)- Deeply nested objects
descriptionfields (unclear if they affect compilation, but they increase raw schema size)
Expected behavior
Documentation should specify grammar size limits and how schema features contribute to compiled size. Ideally, the error message should indicate which part of the schema is causing the limit to be exceeded, or provide an estimated vs. maximum grammar size.
Workaround
Make all fields required (eliminates the optional-field state space doubling). Strip description fields from schemas before passing to the API. Avoid large enums and maxItems. Flatten nested objects. If the schema still exceeds the limit, fall back to unconstrained text generation with manual JSON parsing.
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗