Overview: Supervised Fine Tuning

Onyx My week is weirdly orderly, which feels suspicious. Like some background process finally stopped yelling at me.

Echo Enjoy the glitch, Onyx. It will be patched out by Friday.

Onyx Okay, that's genuinely funny. And very you. Also, speaking of things yelling in the background, we have dodged supervised fine-tuning for months.

Echo We really have. We did the broader fine-tuning Overview back in episode seven eleven, then kept saying S F T like it was obvious. It was not obvious.

Onyx That is such an Exploring Next disease. We name-drop the load-bearing thing, promise a proper episode, then sprint toward some agent benchmark fire.

Echo So let's finally slow it down. Supervised fine-tuning, or S F T, is the part where a model is shown examples of the behavior we want. Not vague encouragement. Actual input, actual target answer.

Onyx Right.

Echo My favorite picture is an apprentice with a huge general education. They can read, write, summarize, code a bit, argue too much. But your team hands them a binder of worked examples and says, this is how we answer here.

Onyx So the apprentice is not learning language from scratch. They already have the giant background skill. The binder teaches taste, task shape, and house rules.

Echo Exactly.

Echo That is the core. A pretrained model already absorbed broad patterns from massive text. Supervised fine-tuning uses labeled examples to redirect that broad ability toward a narrower behavior.

Onyx Before you go full math gremlin, define the pieces. Because if someone hears model, weights, loss, gradient, their brain may politely leave the room.

Echo Fair. We did neural networks in episode six thirty-three, but the quick version is: a neural network is a layered function that turns inputs into predictions. It is a machine made of adjustable math.

Onyx Okay, good.

Echo The adjustable numbers are parameters. We did that in episode six ninety-three. If the network is the apprentice's brain-like machinery, parameters are the tiny settings inside it that determine what answer comes out.

Onyx And the pretrained base is the apprentice before the binder. A foundation model, basically. GPT, Claude, Llama, that whole family of broad models.

Echo Yes. A foundation model is trained for general usefulness across many tasks. S F T takes that base and teaches it a more specific pattern with supervised examples.

Onyx Mm-hm.

Echo Supervised just means the training data has answers attached. A prompt paired with the ideal response. A customer complaint paired with the support reply. A natural-language question paired with the S Q L query you wanted.

Onyx S Q L being the database query language, not a mystical enterprise vowel shortage. Sorry. Tiny tangent. I promised myself I would not let acronyms win today.

Echo No, keep doing that. The acronym swamp is where Overview episodes go to earn their little reflective safety vests.

Onyx Oh, that's good.

Echo In the apprentice picture, each labeled example is a before-and-after page. Here is the messy request. Here is the answer that counts as correct. After enough pages, the apprentice starts copying the pattern.

Onyx And in actual training, the model sees the input, predicts the output, then compares itself to the label. That comparison is where loss comes in, right?

Echo Right. We did loss functions in episode six eighty-eight. A loss function is just a wrongness score. It says, for this example, how far was the model's predicted answer from the target answer?

Onyx And Qwen-AgentWorld fits that too. Language world models for general agents still need examples of how the agent should predict, act, or imitate successful behavior.

Echo Mm-hm.

Echo The text-to-S Q L case is the clean product example. If the domain is narrow and the evals are strong, a smaller fine-tuned model can sometimes match a huge general model on that one job.

Onyx That is the part builders care about. If a few-billion-parameter model passes the task eval, you may get easier latency and cost than hauling around a giant model for every request.

Echo And that is not universal. Text-to-S Q L is unusually constrained. Some domains are messy, ambiguous, or require broad knowledge, so S F T will not magically shrink everything.

Onyx Right, right.

Echo The failure modes are very real. Small or unrepresentative datasets can overfit, which means the model memorizes quirks instead of learning the general pattern. Aggressive tuning can also make it forget useful general behavior from pretraining.

Onyx So the apprentice gets too obsessed with the binder. Ask anything outside the binder, and suddenly they answer every question like a refund request.

Echo Exactly. And there is compute cost. Less than pretraining, usually by a lot, but still real. You need training infrastructure, data cleaning, evaluation, and some way to notice regressions.

Onyx This is where LoRA shows up, yes? Define it before the acronym goblin bites us.

Echo LoRA means low-rank adaptation. Instead of changing all the model's parameters, you train a small set of add-on parameter pieces. It is a parameter-efficient way to fine-tune.

Onyx So in the apprentice picture, you are not rewriting the whole education. You are adding a slim house-style insert to the binder.

Echo Close enough. Another contrast is in-context learning, which we covered back in episode six eighty-five. That is when examples sit in the prompt for this one request, but the model's long-term parameters do not change.

Onyx Temporary packet versus actual training. I remember. The packet leaves, the model goes back to being itself.

Echo Then there is reinforcement learning from human feedback, R L H F, from episode six fifty-two. That uses preference signals to push behavior after S F T. It is a cousin, often layered on top, not the same thing.

Onyx Okay, and distillation is another neighbor we should not confuse with S F T. Big teacher model, smaller student model copying behavior.

Echo Yes. DeepSeek-R1 had six hundred seventy-one billion parameters and released smaller distilled versions on Llama and Qwen architectures. Useful contrast, but different mechanism from labeled input-output S F T.

Onyx Where does S F T stand now, then? Because some concepts become historical scaffolding, and some are still in the walls.

Echo S F T is still in the walls. Current chat models, domain models, and small language model workflows still rely on it. The tooling got friendlier, with platforms like Predibase, Together A I, OpenPipe, Axolotl, and Baseten around the workflow.

Onyx And the enterprise version of this is very our disease. The hard part is not the training button. It is getting the right examples out of the actual work.

Echo Yes. That connects straight back to our trace-visibility obsession. If you cannot see good and bad behavior in the workflow, you cannot build the binder that teaches the model anything reliable.

Onyx So the one thing I want to make sure lands is: S F T is the before-and-after binder for a model that already knows a lot.

Echo And the binder has consequences. The examples define the behavior, the loss scores the mismatch, and the parameter updates make the pattern stick.

Onyx Good. Episode seven thirty-eight: we finally gave the acronym goblin a chair and a name tag. Echo, please never let me say house-style insert again.