Metraipolis › archive › dc_c226335ecdb2
SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management — arXiv:2610.31840
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31840
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:52:52.470Z
- SHA-256 of the archived text
9b3e547814e20a1ee1751831b12dae69fcaa4514695376c26e8ed1b588fccb78- Stable URL of this page
https://metraipolis.com/documento/dc_c226335ecdb2
How to cite
SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management — arXiv:2610.31840 — arXiv (en), https://arxiv.org/abs/2610.31840 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c226335ecdb2 (Metraipolis archive dc_c226335ecdb2, sha256 9b3e547814e20a1ee1751831b12dae69fcaa4514695376c26e8ed1b588fccb78)
BibTeX
@misc{dcc226335ecdb2,
title = {SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management — arXiv:2610.31840},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31840},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c226335ecdb2; sha256 9b3e547814e20a1ee1751831b12dae69fcaa4514695376c26e8ed1b588fccb78},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31840. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tool-calling agents are mostly optimized for accuracy; in deployment, however, their growing cost is latency: every tool call is decoded autoregressively. Speculative decoding accelerates this without changing the output, and retrieved past calls make strong drafts under real use, where each user's behavior tends to repeat. Yet existing systems keep no cross-session memory, or build the draft store once and never update it. We propose SpecMem, a tool-call memory kept alive and managed throughout deployment: verified calls are written back online into bounded per-user stores with LRU eviction, and a side-effect gate defers irreversible calls until verification confirms. On multi-session, multi-user query streams served by three open-weight models, a frozen store steadily loses acceptance while SpecMem holds it: a 27% gap in mean accepted tokens (+11% to +34% across benchmarks, tracking how much novel work arrives after the store is built) and up to 1.74x wall-clock speedup. Ablations attribute the gain almost entirely to keeping the store live; local per-user scoping adds no acceptance but closes a measured cross-user leakage channel, and gating on declared side effects, rather than on retrieval confidence, is what eliminates irreversible mistakes. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.31840 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:48.328Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).