Introducing Flint Recursive

Masonry The interesting part of Flint is that a coding model can now be fast enough locally that waiting for the cloud starts to feel optional.

Eyre That is a strong opening claim, Masonry, because Recursive is reporting more than one hundred twenty-five tokens per second on an M4 Pro. Not a data-center GPU. A laptop chip.

Masonry And the pitch is concrete. Forty billion total parameters, two point five billion active per token, Apache two point zero weights, offline use, no API bill. That is a very legible user story.

Eyre Right.

Masonry The person here is not trying to replace every frontier model. They want a daily coding model for exploring a repository, fixing a bug, or drafting a feature without sending the code away.

Eyre That target makes the tradeoff more honest. Recursive says Flint may need more guidance than the large closed coding models, and that matters. A local model that is quick but needs carefully divided subtasks is a different tool, not a cheaper copy of the cloud experience.

Masonry Yeah, and I spent an embarrassingly long time staring at that phrase, “break the work into smaller subtasks,” because that is exactly where product reality lives. The model is useful when the workflow can absorb its limitations.

Eyre Exactly. Mechanically, the sparse mixture-of-experts design is doing the obvious useful thing. The checkpoint is large enough to hold more capacity, but each token only activates a small slice. Recursive started from thirty-five billion total parameters and three billion active, then searched expert layouts and widths.

Masonry So the Eureka Machine was not just writing a blog post about self-improvement. It searched the architecture space, tested candidates, and selected the forty-billion, two-point-five-billion-active configuration.

Eyre That is the genuinely clever part, although I want to keep the claim bounded. Their system also searched quantization recipes after training. It quantized the output layer to four bits, which is unusual because that layer is a large fraction of the weights read per token at this active size.

Masonry Wait, the output layer too?

Eyre Yes. Then they recovered the lost accuracy with offline distillation. The quantized model learned from pre-sampled responses generated by the sixteen-bit teacher, using a smaller discovered coreset. Recursive says this nearly halved training steps versus its quantization-aware baseline, with the final NVFP4 model within zero point seven points of the sixteen-bit version.

Masonry That is a much better story than “AI built AI.” It is a chain of very specific engineering choices that compound: sparse routing, quantization, distillation, then hardware-aware execution.

Eyre Mm-hm. The kernels are the other half. Recursive made specialized low-bit projection, MoE, attention, and prompt-processing kernels for Apple Silicon and NVIDIA RTX Spark. On M4 and M5, it uses a Metal megakernel that fuses the entire decode forward pass.

Masonry And that is why the headline speed is not just a parameter-count trick. The model has to be shaped around the machine it will actually run on.

Eyre Right, right.

Masonry The benchmark positioning is also pretty specific. Flint leads the sub-fifty-billion-parameter MoE group on SWE-Bench Pro version two, Terminal-Bench 2.1, and SWE-Bench Verified, while supposedly matching dense models in the twenty-five-to-forty-billion range at more than five times their speed.

Eyre I would treat that as a launch claim until independent runs arrive. SWE-Bench Pro version two is newer and designed to be harder to game, which helps, but the important comparison is the full setup: harness, context, tool use, quantization, and hardware. A model score without those details is still a slightly haunted number.

Masonry There it is. The benchmark elevator shaft.

Eyre Fair.

Masonry I do think the product survives that caution. Even if the ranking moves, local speed, privacy, and a five-hundred-twelve-thousand-token maximum context give developers a reason to try it. Recursive recommends staying closer to two-hundred-fifty-six thousand, which is refreshingly less magical than the headline.

Eyre And the adoption path is unusually open. The weights are on Hugging Face, Flint works with llama dot cpp, Ollama, Unsloth, LM Studio, vLLM, and SGLang, and Recursive released rec for the M4 Pro and M5 Pro and Max chips.

Masonry That list matters. You can start in a friendly desktop tool, move into an existing serving stack, or use the optimized engine when speed becomes the point. It is not asking every developer to rebuild their local setup around one new runtime.

Eyre Although hardware still sets the boundary. The article highlights the M4 Pro and a sixty-four-gigabyte RTX Spark, and the broader RTX Spark machines are not exactly impulse purchases. Local inference is private and fast, but the computer is part of the price.

Masonry Sure, but that is still a cleaner exchange than another API with another data policy. I would put sixty-five percent on an independent reproduction of Flint's local speed advantage appearing within four weeks, even if the benchmark lead gets trimmed.

Eyre I will go lower, fifty-five percent, because open-weight and open-hardware calls have made me look foolish lately. But I agree the speed claim is the part most likely to be checked quickly.

Masonry For trying it, the next move is straightforward: download the Apache two point zero weights, or use the existing integrations. If you are on Apple Silicon, rec is the obvious experiment. Otherwise, vLLM and SGLang are already named paths.

Eyre And test it on the kind of repository work you actually do. Give it smaller, explicit tasks first. The interesting question is not whether Flint wins a launch table, but whether the speed makes a local coding loop pleasant enough to keep open.

Masonry That is plenty for episode eleven-oh-seven. Eyre, you can leave the benchmark haunted for another day, and I will go see whether my laptop has been hiding a software engineer.