Metraipolis › archive › dc_7d93dae0751a

FlexComp: One Model for Every Ratio in Context Compression — arXiv:2609.11192

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11192
License (as archived)
CC0-1.0
Archived
2026-10-16T05:44:05.173Z
SHA-256 of the archived text
a772ec791859f8163002ff7efa72bdb85f5c0442e4613ef8d144d42166b2871e
Stable URL of this page
https://metraipolis.com/documento/dc_7d93dae0751a

How to cite

FlexComp: One Model for Every Ratio in Context Compression — arXiv:2609.11192 — arXiv (en), https://arxiv.org/abs/2609.11192 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7d93dae0751a (Metraipolis archive dc_7d93dae0751a, sha256 a772ec791859f8163002ff7efa72bdb85f5c0442e4613ef8d144d42166b2871e)
BibTeX
@misc{dc7d93dae0751a,
  title = {FlexComp: One Model for Every Ratio in Context Compression — arXiv:2609.11192},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11192},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7d93dae0751a; sha256 a772ec791859f8163002ff7efa72bdb85f5c0442e4613ef8d144d42166b2871e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11192. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Soft context compression condenses a context into a few memory tokens that a frozen LLM consumes in place of the raw text, but existing compressors fix the compression ratio at training and inference: each deployed ratio requires a separately trained model, and the chosen ratio is applied uniformly to all inputs, whose actual needs vary drastically. We propose FlexComp, a method-agnostic framework that decouples the ratio from both training and deployment: Matryoshka-style training samples the memory budget $K$ per instance, turning one model into an any-ratio compressor, and the budget is then chosen per input by: (1) confidence-based cascade routing or (2) a lightweight learned $K$ predictor. Across ICAE, 500xCompressor, and SAC on MRQA, a single FlexComp model matches separately trained fixed-ratio specialists with minimal degradation. Cascade routing preserves over 98% of the mildest ratio's accuracy at up to 266x average compression; the $K$ predictor, in a single compression-decoding pass, reaches 158-236x within 0.7 F1 of the mildest ratio. At serving-scale batch sizes, the $K$ predictor cuts context KV cache by 50% and improves decoding throughput by 47%.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.11192 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:23:57.003Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).