Build more responsive apps with the Decisions API
OpenAI’s generally available Decisions API turns bounded model judgments into a dedicated, low-latency interface. The interesting claim is less that classification is new and more that routing, risk checks, and next-action choices can move into an application’s hot path when the output space is deliberately constrained and evaluated.
Transcript
Onyx A ten-times-faster model call sounds nice. But the real pitch in OpenAI’s Decisions API is that your app can stop asking a model to write an essay when it just needs to pick a lane.
Echo Yeah. This is GPT-6 Luna narrowed around bounded outputs: a probability, one choice from a list, or a score on a rubric. That constraint is doing most of the work here, not some mysterious new form of reasoning.
Onyx Exactly.
Echo The article says it can run up to ten times faster than Luna through the Responses API. That makes sense if the service is optimized for a short structured answer instead of generating text, tool calls, and a whole conversational wrapper around a tiny decision.
Onyx I kept getting stuck on their little avatar demo. Someone says something, the transcript updates, and the app picks happy, sad, concerned, or neutral. Kind of silly. But it is a clean way to show why three hundred milliseconds feels different from a full response.
Echo And mercifully, nobody made the avatar write a paragraph about its feelings.
Onyx Oh, come on. You’d reject an expressive face as infrastructure built for its own sake. Still, this is a real product move for anything with lots of tiny forks: route this request, retry that agent step, ask a clarifying question, flag this for review.
Echo Mm-hm.
Onyx That’s the actual argument. Those judgments were already happening inside expensive general model calls. OpenAI is trying to pull them out as a cheap, typed primitive that can sit in the path of an interaction without making the interaction feel sticky.
Echo Technically, it holds up only when the decision is genuinely bounded. A choice endpoint cannot rescue a bad choice set. If your agent needs to decide between continue, retry, and clarify, great. If the real answer is “the workflow is malformed,” you have quietly forced a wrong action into a neat little schema.
Onyx Right.
Echo The confidence field helps, but confidence is not a permission slip. The sensible pattern is the one Box describes: route routine cases automatically, then send lower-confidence certificate-of-insurance cases to people. You still need data from the ugly edges, not just a tidy test set.
Onyx And that’s where I think this changes something practical. The pricing is ten cents per million input tokens, one cent for cached input, and no output-token charge. For high-volume routing, that makes the decision layer easier to justify than making every branch a miniature chat session.
Echo Sure.
Onyx Their early numbers are pointed in that direction. Ramp says AI-cost attribution became four times faster and thirty percent cheaper. Sierra reports under two hundred eighty milliseconds at p95, with ninety percent lower per-call cost and no drop in customer outcomes.
Echo The Datadog example is the one I’d watch hardest. They use it to score the risk of agent-generated shell commands before remediation, and claim forty-times-faster p95 with comparable accuracy. That is a hot-path safety gate, which is useful. But “comparable accuracy” has a lot of room inside it when a rare bad command is the thing you most care about.
Onyx Oof.
Echo I’m not saying don’t use it. I’m saying fast classification changes the shape of oversight, it doesn’t remove it. This is another round of our old judgment-versus-infrastructure fight. A quick decision is valuable only if there’s a real escalation path, logging, and policy outside the model’s prompt.
Onyx Yeah, no, that’s fair. The best fit is probably a team that already knows its recurring fork and can name the safe answers. A support system choosing route, clarify, or escalate. A document pipeline selecting the next queue. Not an agent wandering around with a three-option menu and a lot of misplaced confidence.
Echo OpenAI does give people a concrete starting point: the Decisions Playground, then the developer guide. The GPT-Live hook is also specific. In a Live session, you add a decisions trigger on transcript deltas, and it emits a decision event your app can use immediately.
Onyx Which means somewhere, inevitably, an avatar will decide that a production outage deserves a cheeky expression. We contain multitudes. Mostly badly configured ones.
Echo Stop it. But yes, start with the unglamorous decision before the expressive face. The article is strongest when it stays there.
Onyx All right, Echo. Tiny decisions, real guardrails, and no emotionally adventurous outage avatar. That’s plenty for episode eleven eleven.