Metraipolis › archive › dc_83ceabf577bd

RecToolBench: Benchmarking Recommendation-Specific Tool Orchestration under Fuzzy User Intent — arXiv:2609.30717

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.30717
License (as archived): CC0-1.0
Archived: 2026-10-08T03:12:03.118Z
SHA-256 of the archived text: 953b101a19cdd454b0168f838bd4d53fc3a2500aecdf1b9d1e2664df6822be3f
Stable URL of this page: https://metraipolis.com/documento/dc_83ceabf577bd

How to cite

RecToolBench: Benchmarking Recommendation-Specific Tool Orchestration under Fuzzy User Intent — arXiv:2609.30717 — arXiv (en), https://arxiv.org/abs/2609.30717 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_83ceabf577bd (Metraipolis archive dc_83ceabf577bd, sha256 953b101a19cdd454b0168f838bd4d53fc3a2500aecdf1b9d1e2664df6822be3f)

BibTeX

@misc{dc83ceabf577bd,
  title = {RecToolBench: Benchmarking Recommendation-Specific Tool Orchestration under Fuzzy User Intent — arXiv:2609.30717},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30717},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_83ceabf577bd; sha256 953b101a19cdd454b0168f838bd4d53fc3a2500aecdf1b9d1e2664df6822be3f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30717. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent advances in agentic recommender systems are shifting recommender systems from passive filtering engines to instruction-following agents that use external tools to resolve user intent. However, existing benchmarks often assume explicit user intent, simplified tool environments, or isolated function calls, leaving realistic tool orchestration for recommendation underexplored. To bridge this gap, we propose RecToolBench, a Model Context Protocol (MCP)-based benchmark for evaluating tool-using recommender agents under fuzzy user instructions. RecToolBench contains more than 1,200 executable tasks across three recommendation domains, 13 MCP servers, and 32 tools, spanning single-tool calls, parallel tool calls, sequential tool chains, and hybrid tool orchestration. We construct RecToolBench with a scalable synthesize--fuzzify--judge pipeline that generates executable fuzzy recommendation tasks, and evaluates agent trajectories using rule-based execution checks and rubric-based LLM evaluation. Experiments on representative LLMs show that syntactically valid tool calls do not guarantee successful recommendations. Models struggle with semantic parameter grounding, multi-step evidence integration, and grounded final recommendations, especially as orchestration complexity increases. Our results identify tool orchestration under fuzzy user intent as a major bottleneck for agentic recommender systems. Our data and code are available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-28 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2609.30717 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T02:50:01.732Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).