Metraipolis › archive › dc_5b9467a4aba2
Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents — arXiv:2610.34817
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.34817
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:02:09.092Z
- SHA-256 of the archived text
470f7dd77707df3a6b328d700fe6263d7412ce3b7b4e38b5c77cf77da227033a- Stable URL of this page
https://metraipolis.com/documento/dc_5b9467a4aba2
How to cite
Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents — arXiv:2610.34817 — arXiv (en), https://arxiv.org/abs/2610.34817 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5b9467a4aba2 (Metraipolis archive dc_5b9467a4aba2, sha256 470f7dd77707df3a6b328d700fe6263d7412ce3b7b4e38b5c77cf77da227033a)
BibTeX
@misc{dc5b9467a4aba2,
title = {Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents — arXiv:2610.34817},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.34817},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5b9467a4aba2; sha256 470f7dd77707df3a6b328d700fe6263d7412ce3b7b4e38b5c77cf77da227033a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34817. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) services process massive volumes of user requests, resulting in substantial inference costs. To reduce both cost and latency, providers have introduced prompt caching. However, correctly integrating prompt caching is non-trivial due to heterogeneous provider-specific APIs and continuously evolving caching semantics. To understand prompt-caching mis-integration in downstream applications, we conduct the first empirical study of 120 prompt-caching-related pull requests from 10 popular open-source agent systems. Our findings show that prompt-caching mis-integrations arise in both request-side misconfiguration and read-side usage miscount. More importantly, most mis-integrations fail silently without causing observable runtime failures. Motivated by these findings, we design CACHELINT, which models a prompt-caching mis-integration as an invocation site together with the preconditions required for correct cache integration. CACHELINT locates caching invocation sites with CodeQL and discharges each precondition either against a dated provider capability oracle or with an LLM guard check. CACHELINT achieves 0.71 precision and 0.78 recall in identifying the 120 labeled mis-integrations across the 10 studied agent systems. Our evaluation further demonstrates the effectiveness of CACHELINT in real-world settings, identifying 32 mis-integrations across 571 agent projects with 0.86 precision. These findings have received 21 developer confirmations and led to 18 upstream fixes, including one in Anthropic's official SDK. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.34817 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.941Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).