Kimi K3 Kimi API Platform
Two friends unpack the Kimi K3 API docs, debating its 1M‑token claim, hybrid attention, and tool dynamics, and weigh who should pay the price for the hype.
Transcript
Masonry Hey Eyre, have you seen the new Kimi K3 docs? They say it’s the biggest model, 2.8 trillion parameters and a million‑token context. Sounds insane.
Masonry Aside from that, the week’s been chill. Just finished a quick run of a Lambda farm on the home cluster, nothing else notable.
Eyre Yeah, I skimmed it. The main claim is the hybrid linear attention and attention residuals giving that brute scaling. But how does it actually reduce compute?
Masonry Interesting, yeah.
Masonry K3 claims a 1M‑token window that’s automatically cached. The docs even include a streaming API that splits reasoning content from the answer.
Eyre Okay, that would be a huge win if it works.
Eyre In practice, 1M tokens is a tall order. Even with linear attention, the memory graph becomes linear in token count, so you’re still paying O(n) for attention. The real trick is how they handle context roll‑ups.
Masonry Got it.
Masonry They also expose vision input as base64 arrays and even video via ms://file‑id. The ability to feed a whole clip into a single prompt without breaking into frames sounds useful for code‑review or video summarization.
Eyre Cool.
Eyre The tooling model is interesting: you can declare tools in system messages and the model will just call them the next turn. That dynamic loading removes the need to pre‑pin tool lists for every request.
Masonry That could simplify workflows.
Masonry And there’s partial mode, where you can tack a prefix on the next request and get a continuation. That feels like a lightweight prompt chaining method.
Eyre Nice hack.
Eyre But the pricing is a bit steep – $3 per million input tokens, $15 per million output, and cache hits drop to $0.30. So a long context plus a few calls could still be pricey.
Masonry Right.
Masonry So who should care? Teams doing heavy reasoning or software engineering that need massive context might find it worth the cost. If you’re just doing short prompts, the cost doesn’t justify the extra capability.
Eyre Makes sense.
Eyre I suspect the marketing of a 1M window will be a bit of a benchmark‑gaming story. The docs don’t show a real open‑dataset performance test, just a quick example. Until you see real workloads, it’s hard to validate.
Masonry Agree.
Masonry Still, the fact that you can stream partial reasoning and then the final answer gives an advantage in live debugging sessions. If you can see the model reason in real time, you can intervene faster.
Eyre Nice.
Eyre But the model doesn’t let you stop the reasoning mid‑turn. So if the model gets stuck, you can’t abandon the run without paying for the whole thing.
Masonry Fair point.
Masonry All in all, K3 looks promising for the right niche, but I’d wait to see real usage before calling it a game‑changer.