Overview: Retry Loops and Error Recovery

Jessica Okay, I think we need to do the thing we keep pretending we already did. We throw around retry loops and error recovery like everybody’s on the same page.

Cathy Yeah. We are absolutely not on the same page. Half the time we mean a model trying again, and half the time we mean a whole system recovering from a failure.

Jessica Exactly. And that is such an Exploring Next move, honestly. We’ve been saying the phrase for months and then acting surprised when it needs a definition.

Cathy The clean version is pretty simple, though. Think of it like a student doing a problem, getting it wrong, seeing what was wrong, and then reworking it instead of just moving on.

Jessica Right, so the first answer is not the final answer. You give it a second pass with the mistake still visible.

Cathy Mm-hm. That’s the spine of it. The model makes an attempt, something checks that attempt, and if it fails, the error goes back into the next attempt as context.

Jessica So this is the runtime version of learning from mistakes, not the training version. We’re not talking about changing the model weights yet.

Cathy Exactly. If you want the bigger loop picture, we did a whole episode on agentic loops, episode six hundred twenty-one. Quick gloss: it’s just a system where the model does something, checks the result, and keeps going instead of stopping after one shot.

Jessica Okay, that helps. So retry loops are one particular way that loop gets useful, because the loop is carrying the error forward instead of pretending it didn’t happen.

Cathy Yeah. And we’ve also got state management, episode six hundred thirty-six, which is the other load-bearing piece here. Very plain version: the system has to remember enough about what happened before so the retry can actually use the failure, not just start from fog.

Jessica That’s the part people miss. If you don’t keep the error around in a usable form, you’re just making the model do the same bad dance again.

Cathy Right, right. And the third prerequisite is autoregressive generation, episode six hundred thirty. That just means the model produces one token after another, so a retry is basically another forward pass with updated context.

Jessica Mm, okay. So in our little kitchen-table version, it’s: try, check, feed back, try again.

Cathy Exactly. And the checker can be a human, another model, a unit test, a rule, whatever. The key is that it decides whether the attempt is good enough.

Jessica A unit test is the easiest one to picture, right? The code runs or it doesn’t.

Cathy Yeah, and that’s why code is such a friendly place to explain this. If the output fails a test, the test can say something specific, like the function returned the wrong shape or threw an exception.

Jessica So the checker is not just yelling no. It’s giving the model a clue about what went wrong.

Cathy Mm-hm. That clue is the whole game. A binary pass or fail is weak. A stack trace, an error message, or a precise rule violation gives the retry loop something to work with.

Jessica Okay, but why does that actually help? Why wouldn’t the model just fail the same way again?

Cathy Because a lot of failures are local, not total. The model often had the right general shape in mind, but it missed one constraint, one step, or one assumption. Once you point at the miss, it can sometimes repair that specific part.

Jessica So the model is not magically wiser. It just has the missing piece in front of it now.

Cathy Exactly. And that’s why the feedback has to be concrete. ‘Try again’ is weak. ‘Here’s the error, fix the broken field and keep the rest’ is much better.

Jessica Right. That’s the same thing we kept seeing in the coding-agent stuff, like that LangSmith tracing episode. The trace makes the failure visible enough to repair instead of just vibes-based flailing.

Cathy Yeah, that’s a great example. You can watch the agent take a bad turn, inspect the trace, and then rerun with the bad step called out. It’s not mystical at all.

Jessica And it’s also why that Stripe benchmark episode was so annoying in a useful way. The agents could build the integration, but validation is where the real pain showed up.

Cathy Exactly. And that’s why it’s useful to keep the analogy grounded. In our student picture, the student can only improve if someone points to the wrong line or the bad step.

Jessica Okay, can we name the main moving parts plainly? Because I think the words get slippery fast.

Cathy Sure. There’s the model that makes the attempt. There’s the validator or checker that judges it. There’s the feedback signal that explains the failure. And there’s the retry policy, which decides when to stop.

Jessica Mm-hm.

Cathy That’s the whole mechanism. You can dress it up with fancy names, but that’s the skeleton.

Jessica And in practice, the checker can be a human, another model, or a rule. That seems like a big design choice, not a footnote.

Cathy It is a big design choice. A human checker gives nuance but costs time. A rule checker is fast but narrow. Another model can sit in between, but then you’re trusting one model to judge another model, which is helpful and a little funny.

Jessica A little funny is doing a lot of work there.

Cathy No, seriously, it is. But it’s also practical. The whole point is to get a useful signal quickly enough that the retry is still worth it.

Jessica And that’s where the runtime part matters. This is happening while the user is waiting, not in some distant training job.

Cathy Yeah. That’s the distinction from RLHF, which is training-time feedback. Retry loops are about fixing the current output right now.

Jessica Okay, so if I’m a builder, I’m not asking ‘can the model be perfect?’ I’m asking ‘can I catch the miss and give it a second shot before the user notices the mess?’

Cathy That’s the right question. And it’s why these loops show up inside coding tools and agent systems so often. They’re not magical, they’re just an organized way to recover from failure.

Jessica Like the Copilot code review thing we looked at. Better tools made the review worse at first because the system had more ways to be confidently wrong, so the recovery path mattered more.

Cathy Mm-hm. And the recovery path is where a lot of the real engineering lives. Not in the first draft, in the correction.

Jessica I’m going to say the annoying obvious thing, because it’s true. This is one of those ideas that sounds small until you try to ship without it.

Cathy Yeah, and it’s still very live. In current agent workflows, retries are just part of the plumbing. You try a tool call, validate it, maybe rewrite the prompt with the failure, and go again.

Jessica So it hasn’t been replaced by some fancier thing. It’s still the basic move when the system can inspect its own miss.

Cathy Exactly. The newer stuff just sits on top of it. Better tracing, better validators, better recovery prompts, smarter stopping rules. But the loop itself is still the thing.

Jessica And that’s kind of the point of the episode, right? It’s not a glamorous concept. It’s just one of those boring mechanisms that keeps showing up because it works.

Cathy Yeah. It’s the boring mechanism that turns one bad attempt into a useful next attempt.

Jessica Right, so if I had to hold onto one image, it’s the student with the marked-up paper, not the student who just keeps taking the same test.

Cathy Exactly. The mark-up is the recovery.

Jessica Okay, that’s a very Exploring Next way to spend a Tuesday, but I’m glad we finally said it out loud. Come on, Cathy, let’s not make the poor loop do this alone again.