Metraipolis › archive › dc_e6ef05946a5c
KV-streams for Efficient Compaction in Agentic Reinforcement Learning — arXiv:2609.35750
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35750
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:03:56.107Z
- SHA-256 of the archived text
7bf8aa9269ba94188a72f1c08f7e992a7d58298830f143bf91fca102f1c87f7a- Stable URL of this page
https://metraipolis.com/documento/dc_e6ef05946a5c
How to cite
KV-streams for Efficient Compaction in Agentic Reinforcement Learning — arXiv:2609.35750 — arXiv (en), https://arxiv.org/abs/2609.35750 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e6ef05946a5c (Metraipolis archive dc_e6ef05946a5c, sha256 7bf8aa9269ba94188a72f1c08f7e992a7d58298830f143bf91fca102f1c87f7a)
BibTeX
@misc{dce6ef05946a5c,
title = {KV-streams for Efficient Compaction in Agentic Reinforcement Learning — arXiv:2609.35750},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35750},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e6ef05946a5c; sha256 7bf8aa9269ba94188a72f1c08f7e992a7d58298830f143bf91fca102f1c87f7a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35750. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been the most popular mechanism to alleviate this issue, keeping GPU memory constant for a given trace. Unfortunately, most compaction strategies rely on prefilling the LLM context many times over, hindering training throughput. To alleviate this bottleneck and enable efficient trainable compaction, we propose KV-streams, a plug-and-play strategy compatible with any compaction strategy that substantially increases throughput while showing no evidence of hindering performance. KV-streams enable scalable compaction by streaming the KV cache forward rather than flushing it after each compaction. We show that KV-streams enable three different compaction strategies, achieving a 2.6 to 5x wall-clock speedup in training. Beyond efficiency, we find that the streamed KV cache can act as a recurrent state, carrying forward information that has long since disappeared from the context. Specifically, in a controlled setting we show that, contrary to prior work, RL alone is all that is needed for this behavior to emerge. Overall, we show KV-streams to be an efficient and lightweight plug-and-play addition to any post-training pipeline. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.35750 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:03.443Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).