Structured Outputs

Language models are trained to produce fluent natural language, which is exactly what makes them powerful for conversation — and exactly what makes them unreliable for machine-to-machine communication. When another piece of software needs to parse a model's output — to extract a list of entities, read a confidence score, or pass data to the next step in a pipeline — natural language is the wrong format. A model that sometimes writes '42%' and sometimes 'about forty-two percent' and sometimes 'approximately 42 percent, give or take' is useless to a parser expecting a numeric field. Structured outputs solve this by constraining the model to produce output in a predictable, machine-parseable format, almost always JSON.

JSON Mode and Response Schemas

The simplest form of structured output is JSON mode — an API parameter that instructs the model to always produce syntactically valid JSON. JSON mode prevents the model from outputting prose, markdown formatting, or partial JSON objects, but it does not constrain what keys or values appear. The model might produce {'result': 42} or {'answer': '42', 'unit': 'percent'} depending on context. For many use cases this is insufficient. Response schemas go further: you provide a JSON Schema definition describing exactly the structure you expect, and the model's output is guaranteed to match that structure — every required key present, every type correct, every enum value from the allowed set. This is called constrained decoding or structured generation. Under the hood, the inference engine masks out any token that would make the output non-conformant, guiding the model to produce only tokens that lead to valid schema-conformant JSON. OpenAI calls this feature Structured Outputs (capital S, the product name). Anthropic's tool-use API achieves the same effect through response schemas attached to tool definitions. Google's Gemini API supports response_schema in its generation config. The mechanism differs by provider but the guarantee is the same: you get back exactly the JSON shape you declared.

Here is a concrete example. Suppose you are building a content moderation pipeline that classifies user-submitted text. You need downstream code to act on the classification result. Compare two approaches: Prompt-only approach: 'Classify the following text as SAFE, WARN, or BLOCK, and explain your reasoning.' The model might output: 'This text appears to be safe. Classification: SAFE.' Or it might write: 'I would classify this as WARN because...' Or: 'The appropriate label here is BLOCK.' All three are semantically correct but structurally incompatible — your parser must handle infinite variation. Structured output approach: You define a response schema with two required fields: label (enum: SAFE, WARN, BLOCK) and reason (string, max 100 characters). The model always outputs: {"label": "WARN", "reason": "Contains mildly aggressive language toward a named individual."}. Your downstream code does result['label'] — one line, no parsing logic, guaranteed to work. The structured approach does not reduce the model's analytical capability. It channels the same reasoning into a reliable container.

When and Why Structured Outputs Matter

Structured outputs matter most in three situations. First, when model output feeds directly into code: if a function receives a model's response and calls result['score'] on it, that field must always be present and always be a number. A single malformed response crashes the pipeline. Second, when building multi-step agent systems: the output of one model call often becomes the input to the next. Structural inconsistency compounds — a slightly wrong format at step two can cause a catastrophic parse failure at step five. Third, when you need to audit or store model decisions: a standardized structured record is far easier to log, search, and analyze than freeform text. Structured outputs are also valuable for extraction tasks — pulling entities, facts, or data fields from unstructured text. If you ask a model to extract all dates and dollar amounts from a contract, a schema like {'dates': [string], 'amounts': [number]} gives you clean data in one step. Without a schema, you get prose that requires another round of parsing. The tradeoff: response schemas constrain the output space. Some nuance can be lost if the schema is too rigid. For genuinely open-ended conversation, natural-language output is correct. For machine integration, structured output is almost always correct.