Ep 1170 GitHub 4:47 w/ Edmund & Geffen

Hugging Face and OpenRouter join forces to make open Weight models easy to run

Hugging Face swapped its own inference router for OpenRouter, so every open-weight chat model on the Hub can now be called through the same Hugging Face API and token while OpenRouter quietly picks the best provider on price, speed and reliability per request. It’s a real consolidation move: open weights are now the majority of OpenRouter traffic, and this partnership tries to turn that sprawl into one stable, auditable surface for devs who don’t want to manage providers themselves.

Embed this episode

Paste this on any site — the player is a self-contained iframe with no cookies or trackers.

<iframe src="https://sandrise.io/exploring-next/embed/1170"
  width="100%" height="180" style="max-width:640px;border:0;border-radius:12px;overflow:hidden"
  title="Exploring Next — Episode 1170 audio player"
  loading="lazy" allow="autoplay" referrerpolicy="strict-origin-when-cross-origin"></iframe>
Embed & API docs →
Script GPT-5.1 Voice Speechify Simba 3.5

Transcript

Edmund So Hugging Face just flipped a big switch: all their language-model inference is now going through OpenRouter instead of their own router.

Geffen Yeah, and the wild part is your Hugging Face code basically doesn’t change, but suddenly there’s this eighty–provider routing brain behind every call.

Edmund Which is exactly the pain a bunch of people are in right now, right? You like open-weight models, but actually running them or picking a host is its own job.

Geffen Right. And it lands at a moment where open weights are not niche anymore. On OpenRouter, they went from about twenty seven percent of tokens last October to over seventy percent by late September this year. Weekly open-weight volume grew more than sixty times in that stretch, versus almost ten times for closed models.

Edmund So the story is: open weights are where the volume is going, but they’ve been kind of annoying to operationalize. This move tries to make that boring for you.

Geffen Exactly. OpenRouter routes across six hundred plus models from eighty plus providers. Instead of just picking the lowest list price, it picks the best mix of price, speed and reliability per request, based on live measurements. In their own measurements, that cut capacity errors by about forty six percent and improved time to first token by roughly seventeen percent. If a provider fails, they retry on the next best provider inside the same request.

Edmund Before, Hugging Face’s Inference Providers would pick one host for your request and if that host fell over, tough luck. Now there’s a built-in plan B without you wiring retries yourself.

Geffen And it’s cache-aware. If you send a stable conversation ID in the X H F Session Id header, the router keeps the whole chat on the same provider that holds the prompt cache.

Edmund Also, for people already on Hugging Face, the pitch is very much: run this with the tools you already use. Same token buys you storage buckets, Jobs, Inference Endpoints, and now this routed chat API that speaks Open A I’s chat and responses APIs and Anthropic’s messages API.

Geffen And pricing-wise, Hugging Face says pricing doesn’t change: you pay the provider’s price on your Hugging Face bill, no extra markup. Part of why open weights exploded is cost: on OpenRouter’s numbers, they average about twenty eight cents per million input tokens and one dollar sixty two per million output, versus a dollar twenty seven and six dollars thirty six for closed models.

Edmund Four-ish times cheaper on average is a big lever, even if you’re not chasing the absolute smartest model for every job. The best open-weight model, Kimi K three point one Pro, still trails the very top proprietary stuff on the Artificial Analysis Intelligence Index: around forty six versus sixty five for Claude Fable five point two, but at roughly a seventh of the cost per task.

Geffen There is one real sharp edge here though: the breaking change on custom provider keys. Language-model calls route through OpenRouter and get billed to your Hugging Face account, so you can’t bring your own provider key anymore. You can still pick a specific provider with a model suffix, or disable providers entirely in settings.

Edmund So if your finance team loved that "we pay Provider X directly" story, this forces a conversation. You’re consolidating billing on Hugging Face instead of spreading it across vendors.

Geffen On the flip side, they did real work on transparency. Model pages now have Providers and Metrics tabs: Providers lists every provider OpenRouter can use, with price, speed and uptime over the last twenty four hours. Your usage overview breaks down spend by the provider that actually served your calls, and you can turn off any provider you don’t trust.

Edmund And just to be clear, this only touches language models. Image, video and audio generation keep going straight through Hugging Face’s own inference providers, custom keys and all, same token and API as before.

Geffen So if you’re building today, the immediate move is basically: do nothing and let your existing Hugging Face chat calls quietly get more resilient, or tag sessions with that X H F Session Id and maybe flip a couple of providers off to match your risk tolerance.

Edmund Yeah. And if you’ve been on the fence about open weights because running them felt like a weekend you didn’t have, this kind of kills that excuse. You pick the model on the Hub, drop it into your agent or S D K, and let OpenRouter juggle eighty providers for you.

Geffen I still want to see real-world latency and error logs from people after this cutover, but as a direction it’s very aligned with that question we keep coming back to: does routing consolidate into a few smart layers while models fragment underneath.

Edmund Well, Geffen, at least for Hugging Face users, that question just got a very opinionated answer. Let’s see how it feels once folks have lived with it for a month.