Metraipolis › archive › dc_958ffcd6bcc3

On Device Agentic Operation Caches -- Classifier-Centric NL-to-Action Generation — arXiv:2609.33141

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33141
License (as archived)
CC0-1.0
Archived
2026-10-07T20:56:29.203Z
SHA-256 of the archived text
5f40268eaa942f5a43cbfbbf9b3fdd234703dd8a599ab769199d70ef97c227ab
Stable URL of this page
https://metraipolis.com/documento/dc_958ffcd6bcc3

How to cite

On Device Agentic Operation Caches -- Classifier-Centric NL-to-Action Generation — arXiv:2609.33141 — arXiv (en), https://arxiv.org/abs/2609.33141 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_958ffcd6bcc3 (Metraipolis archive dc_958ffcd6bcc3, sha256 5f40268eaa942f5a43cbfbbf9b3fdd234703dd8a599ab769199d70ef97c227ab)
BibTeX
@misc{dc958ffcd6bcc3,
  title = {On Device Agentic Operation Caches -- Classifier-Centric NL-to-Action Generation — arXiv:2609.33141},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33141},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_958ffcd6bcc3; sha256 5f40268eaa942f5a43cbfbbf9b3fdd234703dd8a599ab769199d70ef97c227ab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33141. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic AI is increasingly being embedded in software applications to provide natural language interfaces to features and functionality. In most cases these agents are powered by enterprise (100+ billion parameter) or frontier class large language models that require substantial computational resources run and depend on cloud hosted inference to handle the task of transforming natural language inputs into actionable software operations. This reliance on cloud-hosted inference introduces substantial network latency on top of LLM inference times, creates data privacy concerns, and, given the costs of running these models, can rapidly escalate expenses associated with supporting agentic features. This paper introduces a novel means of converting the NL-to-Action problem from a generative one into a classification-centric formulation via on-device operation caches. These caches allow an agentic system to handle frequently occurring classes of actions completely on-device -- reducing latency, enhancing privacy, and lowering operational costs. We show that for a classic NL-to-Formula task, generating Excel Formula in response to user requests, this approach reduces total inference cost by 56% when compared to cloud-only model-routing based inference and, on cache hits, reduces the latency to response latency by 5x.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.33141 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.166Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).