The Complete Guide to Tool Selection in AI Agents MachineLearningMastery

Onyx This one made me happy because it’s basically saying your agent didn’t get dumber. You just kept stuffing tools into it until the demo fell apart.

Echo Yeah. And for once that’s not hype theater, it’s a real systems claim. Past a certain catalog size, tool selection becomes the bottleneck.

Onyx Also, quick mood check, I’m weirdly into practical articles this week. No grand theory. Just, hey, your agent keeps calling the wrong thing, here’s why.

Echo You mean you enjoyed a piece where the answer was boring plumbing again. Growth.

Onyx Do NOT do this to me, Echo. We’ve been doing Exploring Next since November and your main hobby is acting shocked when I like something that might actually ship.

Echo Fair.

Onyx The actual argument here is pretty clean. Tool selection is not some last-mile implementation detail. Once the catalog grows, the model seeing every tool on every turn starts causing the failure, so the fix is to narrow the visible set before the model chooses.

Echo Right.

Echo And the article’s strongest move is that it doesn’t pretend a bigger model solves this. It points at prompt bloat, the lost-in-the-middle effect, and schema interference. If you have a pile of similar tools, the model can grab the right name and the wrong arguments, which is a hard failure, not a slightly fuzzy answer.

Onyx That part felt very real. Especially the product trajectory of five tools in the nice demo, then forty after three teams bolt on file ops, calendar, search, C R M, all of it. That is such an Exploring Next disease. The serving layer keeps becoming the benchmark again.

Echo Mm-hm.

Echo It is, and this is close to what we were saying in that routing episode. If the model’s action space is messy, intelligence stops being the constraint. Discoverability is the constraint. I think the article earns that framing.

Onyx Where it really earns it is the RAG-MCP numbers. Retrieval-filtered selection reportedly took tool accuracy from thirteen point six two percent to forty-three point one three percent, while cutting prompt tokens by more than half. That’s not a rounding error. That’s a different operating regime.

Echo Exactly.

Echo I’d still keep the error bars in my head. That benchmark number is strong evidence for retrieval helping, not proof that every production stack will triple. But technically, the mechanism makes sense. Embed the query, retrieve top K tool descriptions, and only then ask the model to choose. You’re shrinking the search space before reasoning even starts.

Onyx And that changes who should care. If you’ve got six clean tools, honestly this is probably overkill. If you’ve got fifty messy ones with overlapping names, this is suddenly product-critical because wrong tool choice reads to users as the agent being flaky or untrustworthy.

Echo Yeah.

Echo My only real pushback is the article’s gate example. The regex and action-keyword filter is fine as a toy. It’ll catch thanks, okay, clarify that. But in practice, short requests like "weather in Lagos" or "latest invoice" absolutely need tools, and long requests can still be pure conversation. So the idea is good, the sample logic is thinner than the article’s confidence.

Onyx I had the same reaction. It’s useful as a cost-saving prefilter, not as a truth machine. Though I do like the deployment order it suggests, because it’s very sane. Start with gate, then retrieval, then routing or planning if the catalog is still gnarly, then fallback logic, then a benchmark harness so you can tell whether you actually improved anything.

Echo The benchmark bit is maybe the most important part, honestly. Otherwise every team does three cherry-picked prompts, sees one nice tool call, and declares victory. If your tool catalog changes every week, you need a standing eval set for selection accuracy, wrong-schema calls, and no-tool-needed turns.

Onyx Right, because otherwise this whole category becomes vibes with a vector store attached.

Echo That’s rude and also correct.

Onyx There’s also a quiet overgeneralization in the article around exact thresholds. It says degradation shows up around ten to fifteen, maybe fifteen to twenty tools in production. I buy that as a rough warning label, not a law. Tool quality matters, description quality matters, overlap matters. Ten excellent tools can be easier than four sloppy ones.

Echo No, totally. And that lines up with the MCP-description-smells point it cites. If the descriptions are bad, retrieval gets worse and direct selection gets worse. So this is not just a count problem. It’s a catalog hygiene problem, which is less glamorous and more true.

Onyx Honestly, I’d put seventy-thirty on this becoming a default pattern in agent frameworks over the next year. Not because the article is magical, just because any team with a real tool inventory runs into the same wall.

Echo I’m a little higher than that. Maybe eighty-twenty on retrieval plus some kind of routing layer sticking. Planning, less sure. A lot of teams do not need a planner, they just need fewer bad tool descriptions and a harness that tells them when they broke selection.

Onyx Which is annoyingly reasonable. Okay, I cannot believe this is how we’re spending episode five ninety-seven, but you were right to like the boring part.

Echo I obviously didn’t sleep on it. Literally.

Onyx Great. Come back when someone invents a new name for the same retrieval stack, and we’ll do this again.