Topic
Speculative Decoding
2 episodes
-
Overview: Decoding Strategy
We finally slow down on decoding strategy, the rule that turns a model's next-token odds into the actual words you see. We use one hallway-and-doors picture to make greedy decoding, sampling, top-k, top-p, beam search, and newer decoding work feel less like magic knobs.
-
Exploring Next Overview: Speculative Decoding
We finally slow down and unpack speculative decoding from the ground up: the draft model, the verify step, and why it can make generation faster without changing the output. We keep it concrete, because that trick sounds like cheating until the mechanism actually clicks.