Topic
Tokenization
22 episodes
-
Overview: Structured Output
We slow down and explain structured output from the ground up: why free-form model text is awkward for software, how schemas and constrained decoding make it usable, and where the format guarantee stops.
-
GitHub FareedKhan Dev/train LLM From scratch: A straightforward method for training your LLM, from downloading data to generating text.
A single-GPU end-to-end LLM training guide lands on GitHub—hand-written PyTorch, pretrain to chat in one repo, plus full RLHF. Vince is giddy; Ava wants to know which corner of the GPU shelf this actually runs on. Build Next shows the exact CLI to kick it off on a T4.
-
Overview: Sequence Modeling
We slow down and finally define sequence modeling, the idea underneath next-token prediction, language models, and a surprising amount of modern AI. We keep it grounded in one picture: covering the next word and training a model to guess what belongs there.
-
Overview: Context Window Management
We finally slow down and explain Context Window Management from the ground up, because we keep hand-waving it whenever agents, memory, cost, and long tasks come up. The whole thing is the fixed-desk problem: what stays on the desk, what gets compressed, and what falls off.
-
Overview: Active vs Total Parameters
We finally slow down on active vs total parameters, because we keep throwing the phrase around like it explains itself. This is us making the difference click: what a model stores versus what it actually uses when it answers.
-
Overview: Router
We slow down on Router, the little decision-maker inside many AI systems that sends each input to the right expert, model, or retrieval path. We use the triage-desk mental model and build from intuition to mechanism, trade-offs, and where routers still matter now.
-
Alibabas Tongyi Lab Releases Qwen Audio 3 0 TTS a Hosted Text to Speech Model in Flash and Plus Tiers Across 16 Languages
Cooper and Miles examine Alibaba's Qwen-Audio-3.0-TTS, comparing its Flash and Plus tiers, multilingual support, voice controls, architecture, hosted-only trade-offs, pricing, and real production use cases.
-
VideoChat3:Fully Open Video MLLM for Efficient and Generalist Video Understanding
VideoChat3 is a fully open-source video multimodal LLM (4B parameters) that tackles three concrete problems: generalization across short/long/streaming video, computational efficiency for video token explosion, and reproducibility through complete open-sourcing. The core innovation is I3D-ViT (Inflated 3D Vision Transformer) plus adaptive frame resolution, which compresses spatiotemporal redundancy early in the pipeline instead of treating each frame as an independent image. Three curated datasets (2M academic + 116K long-form + 617K streaming = 3M samples total) and multi-stage curriculum learning enable the model to handle diverse video scenarios. Jessica sees a shippable foundation for real-world video apps; Cathy pushes on whether the efficiency gains hold under production load and whether the data pipeline's scale claim is reproducible.
-
Overview: Embeddings
We’re finally doing a full pass on embeddings, because they keep showing up under half the things we talk about. We get into what an embedding actually is, why it turns meaning into usable coordinates, and why that little geometric trick sits under so much of modern AI.
-
Exploring Next Overview: Speculative Decoding
We finally slow down and unpack speculative decoding from the ground up: the draft model, the verify step, and why it can make generation faster without changing the output. We keep it concrete, because that trick sounds like cheating until the mechanism actually clicks.
-
Overview: Token Economics
We finally slow down on Token Economics: why tokens are the meter for cost, speed, memory, and product decisions in language models. We keep using the tiny-slip postage analogy until the whole thing clicks, from tokenization to context windows to real API bills.
-
Overview: Natural Language Processing
We keep running into natural language processing everywhere, so we finally sat down and made it the whole point. We walk through what NLP is, why language is such a weird machine problem, and how the field moved from rules to learned representations.
-
Overview: Prompt Engineering
We’re finally doing the overdue deep dive on prompt engineering, the weirdly practical skill of getting language models to do the thing you actually meant. We keep coming back to it because the difference between a flimsy prompt and a good one is often the difference between nonsense and a usable product.
-
Who will own the AI agent economy? | MIT Sloan
MIT’s Ramesh Raskar argues the agent economy’s big wins won’t be in building task-specific agents but in the marketplaces, protocols, and services those agents will need—like identity, discovery, trust, and stablecoin-based micropayments. Project NANDA is racing to keep this ‘internet of agents’ open before corporate consolidation locks it down, but Raskar gives it one-in-ten odds.
-
Overview: Transformer Architecture
We finally sit down and define transformer architecture from the ground up, because we keep throwing the term around like it’s obvious and it really isn’t. We use the attention-as-a-room-of-index-cards picture to make the mechanism click, then connect it to why Transformers became the backbone of modern language models.
-
Overview: KV Cache
We finally sit down with KV cache and make the whole thing click: what it stores, why generation gets faster, and why memory is the bill you still have to pay. We keep coming back to the same working-memory picture so the mechanics stay grounded instead of turning into acronym soup.
-
Overview: Autoregressive Generation
We finally slow down and make autoregressive generation click: the whole thing is just a model writing one token, then using what it wrote to choose the next one. We keep the focus on the loop, the trade-offs, and why that one-step-at-a-time setup is still the backbone of modern language models.
-
Overview: Tokenization
We slow down and explain tokenization from the ground up: how raw text becomes numbered pieces a model can process, why those pieces are usually subwords, and why the tokenizer quietly affects cost, context, language handling, and product behavior.
-
Overview: Context Window
We finally stop hand-waving context window and work through what it actually is, why token count matters, and why bigger windows help and still fail in real use. We keep coming back to the same working-memory picture until it clicks.
-
AI agent tool routing cuts token use 99% | VentureBeat
Cooper and Miles dig into Alibaba's SkillWeaver paper via the VentureBeat write-up, landing on the real claim: tool routing breaks when decomposition vocabulary doesn't match the tool library, and the fix is a retrieval feedback loop that rewrites the plan around actual available skills. They like the systems shape, question some benchmark framing, and agree the practical takeaway is for teams with large tool catalogs, not everyone building simple agents.
-
\ours: Advancing Masked Discrete Diffusion for High Resolution Image Synthesis
Discussion of \(\ours\) (NLD-Image), a masked discrete diffusion model that tackles two core problems in high-resolution text-to-image synthesis: the lack of self-correction in MDMs and the training difficulty with large codebooks. The paper introduces token editing for iterative refinement and Grouped Cross-Entropy (GCE) to alleviate codebook sparsity, achieving SOTA scores on GenEval, DPG, and HPSv3. Hosts debate its product readiness, mechanism soundness, and whether the gains justify the complexity.
-
Anthropic Lead: HTML Increasingly Better Than Markdown at Keeping Humans Engaged in Agentic Loops
Justy and Cody dig into Anthropic's HTML-over-Markdown argument and land on a pretty specific read: this is less a format holy war than an interface fix for long agent workflows where humans still need to steer, review, and stay mentally present.