Overview: Structured Output

Jessica My week has been suspiciously tidy, which means something is wrong. And then someone asked why we keep saying structured output like everyone was born knowing it. Cathy, fair accusation.

Cathy Completely fair. We have absolutely been hand-waving it. So, yeah, let's finally do the slow version: what it is, why it matters, and why it is less boring than it sounds.

Jessica My mental picture is a form at a front desk. If I ask a person, they can tell me a whole story. If I ask software, I need name in this box, email in that box, phone in that box.

Cathy Structured output is telling the model what boxes exist and what each box is allowed to contain. Instead of a charming paragraph, you get a machine-readable object.

Cathy And the reason that matters is painfully practical. A model might write, “Sarah can be reached at this email and phone number.” A human understands it. A program has to scrape it, and scraping prose is where bugs go to reproduce.

Jessica Okay, that's good. Tiny little bug nursery in the paragraph.

Cathy Structured output says, don't make the downstream program guess. Give it a shape like J S O N: customer name, email, phone, each as its own field.

Jessica So the model is still doing the fuzzy language job, like reading the messy support ticket. But the answer has to come back as the form, not as vibes.

Cathy Yes. And that is the bridge. Language models are good at messy input. Software is good at crisp interfaces. Structured output is the handoff format between those worlds.

Jessica Before you go full systems goblin, define the moving parts slowly. Because I can already feel you reaching for token constraints.

Cathy I was reaching politely. A transformer is the main architecture behind modern language models, and it predicts what piece of text should come next based on the context so far.

Cathy Those pieces are tokens. Text gets chopped into chunks the model can score, like words, word parts, punctuation, or spaces.

Jessica So in the form picture, the model is not writing a whole completed form in one magical stroke. It is choosing the next little printed mark, over and over.

Cathy Exactly. Decoding is the rule for choosing among possible next tokens. The model gives probabilities, and decoding decides which token actually gets emitted.

Jessica Your little decoding goblin returns. I cannot believe the gremlin got promoted to prerequisite.

Cathy Stop it. The gremlin has tenure now.

Jessica Okay, that's genuinely funny.

Cathy In normal free-form generation, lots of next tokens are available. If the model starts answering with a paragraph, nothing stops it. With constrained decoding, the system narrows the choices so only tokens that keep the output valid are allowed.

Jessica Wait, what? Like the model literally cannot type the wrong bracket?

Cathy In the stricter versions, yes. If the schema says the output must start with an opening curly brace, the decoder will not allow a random sentence opener. If the next valid thing is a field name, it blocks tokens that would break the format.

Jessica Okay. So the form has physical grooves. The pen can move, but only inside the boxes.

Cathy Nice. And the schema is the drawing of those grooves: field names, data types, required fields, sometimes allowed values. J S O N Schema is the big current standard for that in many L L M pipelines.

Jessica Alternatives include X M L, which is tag-based text, or C S V, which is rows and commas. But J S O N is what everyone keeps reaching for because web software already speaks it.

Cathy The trade-off remains. More complex schemas can add work. Providers expose different controls: OpenAI structured outputs, Claude structured output paths, Gemini structured output, La Plateforme schema-constrained J S O N, plus self-hosted stacks using Outlines or llguidance.

Jessica That list is why this needed its own episode. A newcomer hears five product names and three libraries, but the underlying move is still: draw the form, make the model fill it.

Cathy Yes, and then validate it. Verification checks that an output satisfies explicit rules, like required fields exist, types match, and values fall inside allowed boundaries.

Jessica And if it fails, you have choices. Reject and retry. Ask the model to repair. Relax the schema. Or route the case to a slower path. That choice is product design, not just model trivia.

Cathy Exactly. Also, structured output solves shape, not truth. The model can produce perfectly valid J S O N containing a made-up phone number. The parser will be happy. Reality will not be.

Jessica That distinction is the whole episode for me. The form can be filled out neatly and still be wrong. Neatness is not honesty.

Cathy Yes. If you need factuality, you still need grounding, retrieval, database checks, business rules, or human review depending on the stakes. Structured output makes the answer usable by software. It does not certify the answer.

Jessica This is where the examples help. In document extraction, you might pull invoice number, total, vendor, due date. In support, customer fields. In an agent, the next action and arguments.

Cathy And function calling sits right next to this. If the model needs to choose whether to use a tool and with what typed arguments, that is function calling. The arguments themselves are often structured. Same form idea, attached to action.

Jessica Like that CrewAI review we went through. The agent story only becomes real when handoffs, tasks, and tool calls have typed edges. Otherwise every step is just another paragraph begging to be misunderstood.

Cathy Graph work loves structure because nodes and edges need names, types, and relationships. If the model emits prose, your graph builder guesses. If it emits fields, the pipeline can load it.

Jessica Voice can feel fluid and human, but the moment you want to preserve steps, decisions, or follow-up actions, you want some structured state under the conversation.

Cathy Retries and repair loops are expensive. If constrained output prevents malformed responses, it can save cleanup work. But if constraints slow every generation, you have to measure the net.

Jessica Cathy, that is your natural habitat: the bill arrives and optimism is asked to leave the room.

Cathy I prefer “optimism is asked to bring receipts.” Very different energy.

Cathy JSONSchemaBench has around ten thousand real-world J S O N schemas, which tells you the field has centered on whether systems can handle messy real schemas reliably.

Jessica So where does structured output stand now? Not historical, not a cute trick. This is still live plumbing.

Cathy Very live. In twenty twenty-six, it is part of the normal application stack for model-backed products. Sometimes it is called J S O N mode. Sometimes schema enforcement. Sometimes constrained decoding. The names differ, the form is still there.

Jessica And it has been absorbed into provider features enough that people may stop noticing it. Which is the fate of useful infrastructure: it disappears right when it starts mattering.

Cathy The remaining hard parts are schema design, validation strategy, latency, and truth. What fields should exist? What is allowed? What happens on failure? What checks the values against the world?

Jessica So the one thing I want to keep in my head is: structured output is not making the model smarter. It is making the answer legible to software.

Cathy Yes. It turns the blank page into a form. The model can still misunderstand the task, but at least the receiving system is no longer parsing a tiny novel.

Jessica Tiny novel deletion is an extremely Exploring Next product win. I will take my tidy week and my boxes, thank you.