Topic
Feed Forward Layer
4 episodes
-
Attention Is All You Need
Onyx and Echo dig into the original Transformer paper — what problem it solved, how scaled dot-product attention and multi-head attention actually work, why dropping recurrence was the real lever, and whether the architectural choices have held up across nearly a decade of subsequent work.
-
Overview: Activation Function
We finally slow down and make activation functions click: what they are, why neural networks need them, and why a tiny mathematical kink keeps deep learning from collapsing into one big straight-line machine.
-
Overview: Conditional Computation
We finally slow down and explain conditional computation, the idea we keep casually name-dropping whenever sparse models, routers, and mixture-of-experts come up. We use the same receptionist-and-specialists picture all the way through, so the mechanism, the savings, and the catch actually stick.
-
Overview: Transformer Architecture
We finally sit down and define transformer architecture from the ground up, because we keep throwing the term around like it’s obvious and it really isn’t. We use the attention-as-a-room-of-index-cards picture to make the mechanism click, then connect it to why Transformers became the backbone of modern language models.