Metraipolis › archive › dc_22bf76addc02
SchemaFill: Efficient LLM Tool Calling via Slot-Parallel Speculative Decoding — arXiv:2610.07086
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.07086
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T11:20:14.807Z
- SHA-256 of the archived text
5ca812bf91281a4e4068cef6b98704caf44bd4c2f2a08181f7880ccc43a26658- Stable URL of this page
https://metraipolis.com/documento/dc_22bf76addc02
How to cite
SchemaFill: Efficient LLM Tool Calling via Slot-Parallel Speculative Decoding — arXiv:2610.07086 — arXiv (en), https://arxiv.org/abs/2610.07086 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22bf76addc02 (Metraipolis archive dc_22bf76addc02, sha256 5ca812bf91281a4e4068cef6b98704caf44bd4c2f2a08181f7880ccc43a26658)
BibTeX
@misc{dc22bf76addc02,
title = {SchemaFill: Efficient LLM Tool Calling via Slot-Parallel Speculative Decoding — arXiv:2610.07086},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.07086},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22bf76addc02; sha256 5ca812bf91281a4e4068cef6b98704caf44bd4c2f2a08181f7880ccc43a26658},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07086. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM agents interact with external systems by generating structured tool calls. Given a user request, conversational context, and a catalog of tool schemas, a tool-calling model must select tools and generate their arguments, potentially producing multiple calls in a single response. Standard autoregressive decoding generates these calls token by token, incurring substantial latency for requests involving multiple calls or many argument fields. The explicit argument structure offers opportunities for parallel generation, but later argument values may depend on preceding fields and calls, so independently generated values can differ from the target model's output. We present SchemaFill, a framework for efficient LLM tool calling through slot-parallel speculative decoding. SchemaFill generates future slot values concurrently as candidates, without requiring advance knowledge of the actual call sequence or argument values. Candidates spanning multiple fields and calls are concatenated for verification by the target model under the actual output prefix. Only verified tokens are committed, and the target supplies corrections when candidates disagree. This applies target verification while exploiting parallelism across slots and calls. On Glaive and BFCL, SchemaFill achieves up to a 4.05$\times$ improvement in end-to-end throughput over autoregressive decoding. Code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.07086 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T11:15:24.669Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).