Metraipolis › archive › dc_2ffabf74b5cc

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead — arXiv:2609.10962

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.10962
License (as archived): CC0-1.0
Archived: 2026-10-16T05:26:44.120Z
SHA-256 of the archived text: 09b06764b3b4fc905f4fb351920c80d66321a21f2d6aee6df5e3d20a7ba04a77
Stable URL of this page: https://metraipolis.com/documento/dc_2ffabf74b5cc

How to cite

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead — arXiv:2609.10962 — arXiv (en), https://arxiv.org/abs/2609.10962 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2ffabf74b5cc (Metraipolis archive dc_2ffabf74b5cc, sha256 09b06764b3b4fc905f4fb351920c80d66321a21f2d6aee6df5e3d20a7ba04a77)

BibTeX

@misc{dc2ffabf74b5cc,
  title = {What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead — arXiv:2609.10962},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.10962},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2ffabf74b5cc; sha256 09b06764b3b4fc905f4fb351920c80d66321a21f2d6aee6df5e3d20a7ba04a77},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.10962. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Studies of the Model Context Protocol (MCP) server ecosystem draw their samples in ways that quietly select for servers that work: reference sets, popularity lists, hand-curated frames, or pipelines that repair a server until it starts. We report what an unrepaired probability sample actually contains. From a 24,135-server registry census we draw 400 npm/stdio servers with a published seed and probe each one over the wire. Only 48.8% complete an initialize handshake, against 66.7% for a hand-curated frame measured with the same instrument, and the dominant failure is not missing credentials (13.3%) but servers that never start at all (37.5%). Among the 195 that do run, hard conformance is total: zero fatal JSON Schema violations across 2,766 advertised tools. Optional safety annotations are the real variance, and the tool-level omission rate on a random draw is 58.8% against 41.5% on the curated frame, so curation flatters this figure too. We then compare the tool descriptions these servers advertise against two tool-use benchmark corpora using one method held constant. Real MCP tools show 2.8% near-duplication at cosine 0.70, and all of it lies within single servers: cross-author near-duplication is 0.0% at every threshold tested. BFCL v4 shows 16.7%, of which 16.4 points lie between independently presented tasks. UltraTool shows 0.3%, cleaner than real tools, so this is a property of BFCL and not of synthetic corpora as a class. Separately, 68.8% of raw BFCL rows and 85.6% of raw UltraTool rows are exact name-plus-description repeats, against 0.4% for real MCP, so any statistic computed over these releases without global deduplication measures repetition rather than tools. All figures regenerate from released scripts and a published seed.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.AI).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.10962 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:17:52.101Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).