A Claude Code Skill for MCP Tools: Find the Ones Your Agent Picks Wrong

Your MCP server works. The handlers are correct, the tests are green, and the agent still reaches for the wrong tool — or reaches for the right one and calls it with arguments you never intended. That is not a bug in your code, and no amount of handler coverage will surface it, because the agent never sees a handler. It picks a tool as a pure function of the text you show it: name, description, inputSchema. If two tools read alike, if a description restates its own name, if one God-Tool hides four verbs behind an untyped action string, the model guesses — confidently, and differently each run. An MCP skill for Claude Code is a fixed audit of that surface. This page is about the one AgentSource ships for the job — the MCP Tool Reliability Auditor — and it is free, with a captured field report you can read before you claim it.

The surface your tests can't reach

There are two descriptions of every MCP tool. One is the handler, which your test suite exercises. The other is the definition, which is the only thing the model is ever shown. They are maintained together and verified separately, and almost nothing checks the second one.

That gap has a specific shape, and the skill's taxonomy names eleven axes of it. The ones that move correct selection most:

• Cross-tool collisions. Two live paths to the same operation. Each description can be individually well-written and the pair still splits calls unpredictably — which is why a per-description linter walks straight past the single strongest cause of wrong selection.
• Missing when-to-use text. Nothing marks the boundary between this tool and its neighbour, so the agent infers one.
• God-Tools. An untyped action: string forking four branches, one of them irreversible. One schema and one annotation set cannot honestly cover reads through hard deletes.
• Undiscriminated errors. A handler that returns the bare string "error" on failure. The agent cannot tell not-found from unauthorized from already-deleted, so it cannot decide between retrying, re-authenticating, and giving up — and a bare string risks being read as data rather than as a failure at all.
• Unguarded mutations. A destructive operation with no destructiveHint and no dry-run. The skill scores that as an auto-fail, because an agent with no consent signal is a selection problem before it is a security one.

Each axis ships with detect-and-fix guidance for Python (FastMCP or mcp), TypeScript (@modelcontextprotocol/sdk), or a live tools/list capture when you only have the wire format to work from.

Rewrites arrive as diffs, not overwrites

The scorecard is the cheap half. The part that takes judgement is the rewrite, and the skill emits every one as a diff you approve rather than editing your server underneath you.

It also names its own tradeoffs instead of reflexing them. When the fixture's God-Tool had to be broken up, the skill put both options on the table with the cost attached: split into separate tools — cleaner selection and honest per-tool annotations, at five tools instead of three, a net +2 on every tools/list your context pays for — or constrain, folding the safe verbs back behind one action: enum[...] and keeping the token budget tight. It recommended split for that fixture and said why, and it said what would make constrain the right answer instead. A skill that silently picks one and calls it best practice is hiding the decision you are actually making.

The field report: a measured result, published with its own caveat

The proof is not the feature list. It is the captured run, free to read on the product page.

The fixture was a three-tool CRM server seeded with planted defects. Phase 1 scored it and found the architecture bug first: manage{action:"delete"} and a standalone delete_contact were two live paths to the same hard delete, with a second overlapping pair on the search side. Two auto-fails — the unguarded delete branch and the opaque "error" return — were gated to be fixed before anything else. Phase 2 resolved the delete duplication by removing the branch entirely rather than just disambiguating its wording, on the grounds that the two were never legitimately separate operations.

Phase 3 is the part most tooling skips. A fresh agent (claude-sonnet-5, tool order randomized per trial) was shown only the definitions against a stub executor and asked to pick one tool for each of six gold-labelled tasks, five trials per task, on the original defs and then the rewritten ones.

Arm	Correct selection	Confusion
Before (3 original tools)	28/30 — 93.3%	get_contact → manage on "find by email"; manage → NONE on "search by name"
After (5 rewritten tools)	30/30 — 100%	none

+6.7 points, N=5 per task. And the report immediately tells you not to be impressed by it: that is a small-N check on one three-tool fixture, a capable model already routes even the poor originals about 93% of the time, so the selection headroom was small by construction. It states plainly that the larger findings were in calling and safety — the invented data shape, the hard delete with no preview, the opaque error — none of which a selection eval scores at all.

The most useful paragraph in the whole report is a method note admitting a mistake. An earlier arm labelled "remove Dana Lee" — a name, no id — with delete_contact as the correct answer. The agent picked search_contacts instead, because the rewritten delete_contact now refuses a name and demands a resolved contact_id. That scored as a miss. It was the eval catching safer behaviour and a mislabelled task on our side, and it is the reason the confusion matrix, not the headline rate, is the output that matters.

Running it without spending anything

The selection eval makes real model calls, so it costs a little. A static-only mode runs the full scorecard and the rewrite library at zero spend, and it is honest about the consequence: no run, no lift number. The skill's standing rule is that no selection figure reaches the report unless a real run produced it, quoted with N, model, and spread. If the eval comes back flat, the report says flat and the rewrite gets another pass.

That rule is also why the published figure above is 6.7 points and not something rounder. You are buying the measurement on your model and your tool surface, not a promised percentage.

It's free — the field report is the point

The MCP Tool Reliability Auditor is $0, claimed with an email. It is the one skill on the shelf given away, for a straightforward reason: the argument this whole depot rests on is that you should be able to read a skill's real output before you decide whether it is worth anything, and this is the cheapest possible way to test that claim. Read the captured run, disagree with it if you like, and you have lost nothing.

If it earns its keep, the discipline it applies — measure the fix, publish the caveat, never assert a number a run did not produce — is the same one behind Deep QA Audit Loop ($9), which loops on an application until zero critical bugs remain rather than stopping after a set number of rounds.

One limit worth stating plainly: this audits definitions, not architecture. If your server exposes the wrong operations for the job, better descriptions will get the agent to the wrong tool more reliably. What it fixes is the far more common case — a sound set of operations the model cannot tell apart. Read the field report first, then decide.