Metraipolis › archive › dc_1371e71d3f87
Koa-action: Fast and Consistent Structured Decision Making with Generative LLMs — arXiv:2609.36115
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36115
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T17:28:42.979Z
- SHA-256 of the archived text
7fa01e2419e93b32eb3b1d4113b90edcab461977e1174ffbf22b0e6fc7d6ba7b- Stable URL of this page
https://metraipolis.com/documento/dc_1371e71d3f87
How to cite
Koa-action: Fast and Consistent Structured Decision Making with Generative LLMs — arXiv:2609.36115 — arXiv (en), https://arxiv.org/abs/2609.36115 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1371e71d3f87 (Metraipolis archive dc_1371e71d3f87, sha256 7fa01e2419e93b32eb3b1d4113b90edcab461977e1174ffbf22b0e6fc7d6ba7b)
BibTeX
@misc{dc1371e71d3f87,
title = {Koa-action: Fast and Consistent Structured Decision Making with Generative LLMs — arXiv:2609.36115},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36115},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1371e71d3f87; sha256 7fa01e2419e93b32eb3b1d4113b90edcab461977e1174ffbf22b0e6fc7d6ba7b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36115. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-critical applications. Existing prompting and constrained decoding produce verbose, multi-token outputs that require expensive token-by-token generation, while encoder-based models achieve faster inference but sacrifice task flexibility. We propose Koa-action, a framework for low-latency atomic actions -- fast, single-step decisions such as classification, semantic endpointing, Boolean checks, and scoring -- formulated as constrained generation with single-token outputs. By introducing atomic label tokens and applying supervised fine-tuning, our method reduces classification to a deterministic one-step decoding problem. Across standard benchmarks, Koa-action delivers competitive accuracy with consistently low and stable latency. On a production intent-routing benchmark, Koa-action reaches 85.5% accuracy -- competitive with the strongest frontier models (Claude-4.8-Opus, Gemini-Pro-3.1) and ahead of GPT-5 and Gemini-2.5-Pro -- while answering in about half a second, several-fold faster than every frontier model (up to ~7.5x at the median) under identical serving conditions. Against the dedicated single-token system Jev/TypeSafe, Koa-action is competitive on accuracy and faster at the median, while also handling multimodal inputs and multi-label outputs that single-label text systems do not. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.36115 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T16:59:48.101Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).