Topic
Activation Function
6 episodes
-
Safety Fine Tuning Suppresses Mind Attribution and Spiritual Belief in LLMs
Justy and Cody discuss a new Google research paper revealing that safety fine-tuning—specifically the effort to stop LLMs from claiming they are conscious—accidentally suppresses their ability to attribute minds to animals or natural objects and reduces their 'spiritual' beliefs, shifting them away from human-like sociological distributions.
-
Overview: Model Interpretability
We slow down and make model interpretability actually click: what it means to explain a model, what the main tools can and cannot show, and why the difference between a useful explanation and a comforting story matters.
-
Overview: Activation Function
We finally slow down and make activation functions click: what they are, why neural networks need them, and why a tiny mathematical kink keeps deep learning from collapsing into one big straight-line machine.
-
Overview: Sparse Activation
We finally sit down with sparse activation and make the idea click from the ground up: why only part of a model wakes up on each input, how routing makes that happen, and where the real trade-offs show up. We keep it concrete, because this one has been lurking under a lot of the stuff we keep talking about.
-
Overview: State Management in Language Models
We finally do the episode we keep circling back to: state management in language models. We walk through the idea from the ground up, using the cache-and-notes picture to show why models don’t have to recompute everything every token, and where that trade-off starts biting.
-
Overview: Neural Network
We finally slow down on neural networks: what they are, how the little adjustable pieces learn from examples, and why this basic idea sits underneath so much of modern A I.