OpenAI has unveiled Structured Outputs, a new API feature that guarantees 100% adherence to developer-supplied JSON schemas in model responses. This update addresses a long-standing challenge in AI application development, where inconsistent output formats often hindered smooth integration with existing systems.
The feature works by constraining the model to generate only valid tokens according to the provided schema, using a technique called dynamic constrained decoding. This approach converts JSON schemas into context-free grammars, allowing for more complex and recursive data structures than previous methods.
Structured Outputs is available in two forms:
- Function calling: Developers can enable strict schema adherence for tool definitions, compatible with models from GPT-3.5-turbo-0613 and GPT-4-0613 onwards.
- Response format option: A new json_schema parameter allows for structured responses outside of function calls, currently available for the latest GPT-4o models.
OpenAI has also updated its Python and Node SDKs with native support for Structured Outputs, simplifying implementation for developers using these languages.
OpenAI’s Structured Outputs also introduce a safety mechanism, allowing the model to programmatically detect and handle unsafe requests. If the model refuses an unsafe request, a refusal string value will be included in the API response, making it easier for developers to identify and manage such instances.
The company has also introduced a new model, GPT-4o-2024-08-06 which achieves perfect scores in complex JSON schema following tests. This model comes with significant price reductions:
- 50% lower input costs ($2.50 per million tokens)
- 33% lower output costs ($10.00 per million tokens)
- 50% overall cost reduction compared to the previous GPT-4o model
By ensuring consistent and accurate JSON outputs, developers can build more robust applications that interact seamlessly with their existing systems. For example, Structured Outputs can be leveraged in applications such as:
- Dynamically generating user interfaces based on user intent.
- Separating final answers from supporting reasoning or commentary, improving response clarity.
- Extracting structured data from unstructured inputs, such as to-dos and due dates from meeting notes.
While Structured Outputs offers improved reliability, OpenAI notes some limitations. The feature supports only a subset of JSON Schema, and the first request with a new schema may incur additional latency for preprocessing. However, subsequent requests using the same schema will be faster.
For more information and to start using Structured Outputs, developers can refer to OpenAI’s official documentation.
