Metraipolis › archive › dc_950d4bdabe61
Long-Context Demonstration Selection Using State Space Models — arXiv:2609.17888
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17888
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T13:49:50.324Z
- SHA-256 of the archived text
b7049e6c0525582fa1c1925e03944af3d0fe66beac4528e2d3980baf8daff0fd- Stable URL of this page
https://metraipolis.com/documento/dc_950d4bdabe61
How to cite
Long-Context Demonstration Selection Using State Space Models — arXiv:2609.17888 — arXiv (en), https://arxiv.org/abs/2609.17888 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_950d4bdabe61 (Metraipolis archive dc_950d4bdabe61, sha256 b7049e6c0525582fa1c1925e03944af3d0fe66beac4528e2d3980baf8daff0fd)
BibTeX
@misc{dc950d4bdabe61,
title = {Long-Context Demonstration Selection Using State Space Models — arXiv:2609.17888},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17888},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_950d4bdabe61; sha256 b7049e6c0525582fa1c1925e03944af3d0fe66beac4528e2d3980baf8daff0fd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17888. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This problem is closely related to in-context learning and language model inference. Since the inference cost of a transformer model scales quadratically with sequence length, the selection problem becomes especially challenging in a long-context scenario. In this paper, we tackle this problem by building on state space models (SSMs), which require only linear inference time given the input. Our approach involves two algorithms. The first learns a small set of SSMs through distillation of a (trained) transformer model. We partition all the layers into consecutive groups. Then for each group, we estimate a separate state space model to replicate the input-output behavior within the adjacent layers. Second, we map the distilled model outputs to a small set of tokens, and apply these embeddings for demonstration selection in downstream applications. We perform extensive experiments in both synthetic and real-world datasets to validate our approach. We demonstrate that the distilled SSMs only incur an approximation error of less than $0.7\%$ relative to the true output. In downstream evaluation, we show that on several text classification and reasoning tasks, our approach reduces FLOPs by $14.2\times$ and improves accuracy by $6.48\%$ relative to baseline demonstration selection methods. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.17888 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:48:24.398Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).