I put my local LLM on Tailscale, and now I can use it from the other side of the world
A local LLM becomes genuinely portable when a private mesh network makes its home endpoint behave like a normal provider connection. The interesting product shift is not remote inference itself, but removing the network friction that keeps self-hosted models trapped on one LAN.
Transcript
Justy The neat part isn't that someone ran a three hundred twenty billion parameter model at home. It's that their phone treats it like a provider from anywhere.
Cody Yeah. Adam Conway's actual claim is smaller and better: Tailscale turns a LAN-only OpenAI-compatible endpoint into a private, roaming service without opening it to the public internet.
Justy Right. That is the moment this stops being infrastructure as a hobby and becomes a product experience. RikkaHub on the phone and Pi on the laptop just point at the same vLLM endpoint.
Cody Mm-hm.
Justy And I like that he keeps the home address through subnet routing, so he can use the same address at home or away. That is such a tiny detail, but it removes the annoying mode switch.
Cody It also means Tailscale is doing the boring work people usually botch: private addressing, key exchange, and finding a direct path where it can. No port forwarding, no public reverse proxy with a hopeful password taped onto it.
Justy We have somehow made it almost a year doing this show, and we're still excited by coordination machinery.
Cody Because the coordination machinery is where the pain lives. Also, a VPN that politely gets out of the way on the home LAN is more emotionally mature than half the agent frameworks we've discussed.
Justy Okay, that was unfairly specific. But it's why I think people with an always-on home server should care. Their expensive hardware stops being a thing they can use only when they're sitting near it.
Cody Sure.
Justy His Portugal example is the practical proof. He transcribed audio locally with Whisper, sent the transcript home for cleanup, and the distance was close enough that it felt normal.
Cody Taiwan exposed the limit, though. The first token took noticeably longer, then streaming settled down. That tracks technically: a mesh can make routing convenient and encrypted, but it cannot repeal propagation delay across continents.
Justy Exactly. But for chat, coding help, transcription cleanup, maybe long-context document work, a slower first token can be fine if the model and the context are already yours. He says the setup keeps the same five hundred twelve thousand token context wherever he is.
Cody I do want to put one guardrail on his security framing. Tailscale membership is a solid access boundary, but his vLLM endpoint has no API token, so any compromised or over-permitted Tailnet device gets the whole endpoint.
Justy Yeah, and subnet routing makes that trade more consequential because it can expose private-network addresses to the mesh. For one person's tightly managed devices, I can see the convenience winning. For a shared Tailnet, I'd want ACLs and app-level auth too.
Cody Right, right.
Cody Tiny calibration note: we read the piece, but we didn't independently verify every implementation detail or chase every newest number. If somebody's making a hardware or security decision on specifics, check the primary docs.
Justy Fair. Still, the article doesn't need the exact DGX Spark setup to hold. He also says Qwen 3.8 27B works on his home server, which is the more important evidence: the pattern survives after the flashy cluster disappears.
Cody That's the technical argument I buy. vLLM exposes an OpenAI-compatible API, and Tailscale gives trusted devices a route to it. RikkaHub and Pi don't need to know they are talking across countries, only where the endpoint lives.
Justy And that changes the self-hosting pitch. Privacy isn't just a slogan about prompts not becoming training data. It's the ability to use your own model for real mobile workflows without hauling a little inference appliance around.
Cody Hm.
Cody The costs don't vanish, obviously. The two Spark machines stay powered on, and Android allows only one VPN connection at a time, so another VPN can force a choice. Hosted APIs remain easier when you don't already own the box.
Justy Which is why this isn't a mass-market overthrow of hosted models. It's a very good unlock for the people who already have a server running, care about private inputs, and keep hitting the same workflow from multiple devices.
Cody If I were copying the stack, I'd start boring: run the model behind vLLM, add Tailscale to the server and client devices, then point Pi or RikkaHub at that private endpoint. Test it on mobile data before declaring victory.
Justy That's a healthy amount of Cody skepticism. Keep the endpoint private, test the latency, and please don't make your home GPU discoverable by accident.
Cody A modest goal for episode one thousand eight.
Justy Honestly, I'll take it. Remote access that feels boring is the whole win, Cody.