NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning | NVIDIA Technical Blog
NVIDIA drops Nemotron 3.5 Super VL — a 120B-parameter open-weight multimodal MoE model built for specialized enterprise agents. Jessica and Cathy dig into the real argument: post-training on your own data beats the frontier on your specific task, and the efficiency story (40x inference cost reduction, 10x latency) is where the product actually lives.
Transcript
Jessica Okay so NVIDIA dropped Nemotron 3.5 Super VL today, and the headline number everyone's going to quote is 40x lower inference cost. But I actually think that's burying the real argument.
Cathy Yeah, agreed — the 40x is a post-training result, not a base model result. Scale AI ran it for Skydance's vendor-credit workflow specifically. That's not a benchmark, that's a production task with a before-and-after. Which is actually more interesting than the benchmark.
Jessica Right, and that's the thesis. It's not 'we built a model that beats GPT.' It's 'take this 120B MoE base, post-train it on your receipts, your return policies, your threat intel — and you beat a frontier model on YOUR thing at a fraction of the cost.' That's a genuinely different pitch.
Cathy It is. And the architecture is worth a second — 120 billion total parameters, but only 12 billion active per token because it's mixture-of-experts. They're also running LiLiCorr speculative decoding, NVFP4 quantization, and multi-token prediction. That's a layered inference stack on top of a hybrid Mamba-Transformer backbone, which handles long-context without blowing up the KV cache.
Jessica And the context window is up to one million tokens, which for document intelligence agents is actually meaningful — you're not chunking a 200-page contract anymore.
Cathy Sure, in theory. One million tokens is the ceiling, not the floor, and most of the efficiency claims are measured on specific harnesses. The 5x throughput number comes from Artificial Analysis benchmarks on Baseten. One endpoint, one eval setup. I'm not saying it's fake — I just want to know what it looks like when Ramp's actual receipt volume hits it.
Jessica Okay, that's fair. Though Ramp's number is interesting on its own — 10x lower latency and 50% higher throughput for receipt-to-structured-JSON. That's a financial ops company saying their pipeline got faster. And the AI21 result — 41% accuracy improvement on threat intelligence — that one's on a defined classification task, which is actually measurable.
Cathy The directionality is real. And now you've got open weights on Hugging Face, GGUF via Unsloth, NIM for managed deployment, llama.cpp and Ollama support — the distribution story is genuinely there. NeMo AutoModel for SFT and LoRA, NeMo RL and NeMo Gym for reinforcement learning, 40 vision-language datasets dropped alongside the model. The question for me is whether most orgs actually have the data and the post-training discipline to use any of this.
Jessica Oh, that's the real question.
Cathy Because the partners they name — Scale AI, Ramp, AI21 — those are not representative of a median enterprise ML team. Prime Intellect built 2,500 verified training tasks just to get the long-horizon agent behavior right. That's 11 hours on 128 GB200 GPUs. Who else is doing that?
Jessica I think that's the point where NVIDIA's betting the NeMo tooling lowers that bar. The cookbook, the datasets, NeMo Data Designer for chart QA and document understanding. That's the pitch: you don't have to be Prime Intellect to start. And the multimodal piece actually matters for the agent story — same model reads a chart, connects it to a table in the same document, writes the summary. No context handoff between a vision model and a text model.
Cathy I hope that's true. Though I'd note the Hermes Bench and LangChain evals where they beat Nemotron 3 Ultra — those are NVIDIA's own harnesses. I'd want to see that hold on something they didn't build.
Jessica EXO Labs placed it on the Pareto frontier for accuracy and task completion time on DGX Spark. That one's at least external.
Cathy Fair. DGX Spark is still an NVIDIA platform, but EXO Labs isn't NVIDIA. I'll take it as weak external evidence. And honestly — given my recent track record on open-weight calls, I'm going to be more careful here than instinct says.
Jessica Oh, NOW you're humble.
Cathy Look, three misses in a row on open-weight predictions will do that to you. I'd put maybe 55-40 that clean independent reproduction of the 5x throughput claim — not on Baseten, on a different endpoint — comes in lower within the next three or four weeks. Not zero, just lower than the headline.
Jessica I'll take the other side at 60-40 it basically holds, because the MoE efficiency story is mechanistically sound and the inference stack is layered enough that the gains are real. The question is just how much the endpoint matters.
Cathy Yeah, we'll see.
Jessica Alright — if you want to poke at it yourself, the weights are on Hugging Face as Nemotron 3.5 Super VL 120B, there's a usage cookbook on GitHub, and NIM is the managed path. Good enough place to land, Cathy.