Metraipolis › archive › dc_53ecd3182e6b
SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use — arXiv:2611.03982
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03982
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:29:37.014Z
- SHA-256 of the archived text
94e3eed1ca7f675f4f89320f4ebbb17077ec2e04b8316106eca9efeccac36e95- Stable URL of this page
https://metraipolis.com/documento/dc_53ecd3182e6b
How to cite
SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use — arXiv:2611.03982 — arXiv (en), https://arxiv.org/abs/2611.03982 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_53ecd3182e6b (Metraipolis archive dc_53ecd3182e6b, sha256 94e3eed1ca7f675f4f89320f4ebbb17077ec2e04b8316106eca9efeccac36e95)
BibTeX
@misc{dc53ecd3182e6b,
title = {SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use — arXiv:2611.03982},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03982},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_53ecd3182e6b; sha256 94e3eed1ca7f675f4f89320f4ebbb17077ec2e04b8316106eca9efeccac36e95},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03982. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Enterprises increasingly want LLM agents to automate standard operating procedures over their software stacks -- mapping each step of a procedure to the right call among thousands of internal APIs, without hand-authored step-to-API mappings. The core difficulty is API selection at scale: choosing and composing the correct tools over a long horizon, judged only by whether the overall task succeeds. Current tool-use benchmarks do not test this regime -- they expose only a handful of tools, evaluate single-step invocation, or grade multi-step behavior with subjective LLM-as-a-judge scores that penalize valid alternative trajectories. We introduce SLATE (Synthetic Large-scale API Toolkit for E-commerce), a large-scale benchmark that tests large-toolset, long-horizon API selection in a realistic enterprise domain. SLATE couples (i) queries decomposed into hierarchical, tool-grounded procedures over a ~1,000-tool e-commerce library, (ii) a deterministic, context-aware simulator that returns recorded outputs for matching calls and default responses otherwise, and (iii) an end-to-end task-success metric -- a task succeeds when it reaches every reference outcome -- that credits diverse-but-valid trajectories while remaining fully automatic and reproducible. Experiments with six open decision models and the language models they are built on, from 3B to 12B parameters, show that even the best model completes only 5.0% of the tasks, establishing SLATE as a challenging testbed for reliable enterprise tool use. We release SLATE, its simulator, and the evaluation harness at this https URL Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.03982 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:26:47.237Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).