Metraipolis › archive › dc_0ca18ee910a2

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents — arXiv:2610.41905

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41905
License (as archived): CC0-1.0
Archived: 2026-11-02T04:46:31.382Z
SHA-256 of the archived text: 44e58ac186211573ad59181e8dcb834bdbf445ecfbcbecd186362e32d21b7170
Stable URL of this page: https://metraipolis.com/documento/dc_0ca18ee910a2

How to cite

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents — arXiv:2610.41905 — arXiv (en), https://arxiv.org/abs/2610.41905 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0ca18ee910a2 (Metraipolis archive dc_0ca18ee910a2, sha256 44e58ac186211573ad59181e8dcb834bdbf445ecfbcbecd186362e32d21b7170)

BibTeX

@misc{dc0ca18ee910a2,
  title = {Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents — arXiv:2610.41905},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41905},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0ca18ee910a2; sha256 44e58ac186211573ad59181e8dcb834bdbf445ecfbcbecd186362e32d21b7170},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41905. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An agent that must call several tools to fulfill a request fails when the tool retriever omits any one of them, yet tool retrievers are mostly built and evaluated on queries requiring a single tool. We compare representatives of six retrieval structures (flat lexical and dense retrieval, LLM query decomposition, LLM-navigated hierarchies, server-then-tool matching, dependency graphs, and trained set-level retrievers) across two multi-tool testbeds: 345 AppWorld queries with expert dependency labels and 103 MCP-Bench tasks over 257 MCP tools. We measure completeness@k, the share of queries whose entire required tool set appears in the top k, alongside the LLM calls, tokens, and latency that each structure incurs at retrieval time. A motivating analysis on all 7,961 ToolRet queries shows that aggregate completeness mixes two regimes: completeness at 10 is .58 to .67 on single-target queries and .08 to .17 on queries with four or more targets, and the ordering of retrievers on single-target queries reverses on multi-target ones. Across the two testbeds, none of the compared structures that is untrained on the testbed and makes at most one LLM call per query completes more than 10% of AppWorld queries at k = 16. The two structures that do better incur different costs: a set-level retriever trained on 75 to 90 in-domain queries reaches .33 on AppWorld and .58 on MCP-Bench with no LLM calls, while an LLM-navigated hierarchy reaches .30 and .40 with a median of 31 to 38 LLM calls and about five minutes per query. Their failure modes also differ: the trained retriever mostly ranks the missing tools just below the cutoff, whereas the hierarchy finds the target API and stops before collecting its prerequisites. In a downstream check with the official AppWorld agent, tasks with a complete retrieved set succeeded 21 of 21 times, whereas incomplete ones succeeded 2 of 39 times.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.41905 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T04:42:26.942Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).