Metraipolis › archive › dc_089a99b520b0
Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering — arXiv:2610.31135
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31135
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-25T08:34:51.188Z
- SHA-256 of the archived text
bdac327915ed6bab9fd1d844b239d791c33281cb8e7f9ab0715bdeb5ae5829ec- Stable URL of this page
https://metraipolis.com/documento/dc_089a99b520b0
How to cite
Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering — arXiv:2610.31135 — arXiv (en), https://arxiv.org/abs/2610.31135 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_089a99b520b0 (Metraipolis archive dc_089a99b520b0, sha256 bdac327915ed6bab9fd1d844b239d791c33281cb8e7f9ab0715bdeb5ae5829ec)
BibTeX
@misc{dc089a99b520b0,
title = {Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering — arXiv:2610.31135},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31135},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_089a99b520b0; sha256 bdac327915ed6bab9fd1d844b239d791c33281cb8e7f9ab0715bdeb5ae5829ec},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31135. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long-context models often use answer-bearing evidence less effectively when it appears away from a prompt boundary. We compare this position bias across Transformer, state-space, and hybrid Mamba architectures by moving one gold document through a fixed set of nine distractors. The evaluated Transformers show a beginning advantage that the evaluated state-space models lack, while both families favor evidence near the end. A matched pure-versus-hybrid Mamba comparison points in the same direction but remains statistically uncertain. The family difference appears among models that can answer the task. Transformer primacy reappears on synthetic retrieval, where state-space ceiling prevents a family comparison. A corpus swap within one state-space architecture changes overall accuracy without a detectable change in curve shape. Prompt changes can create or nearly remove a beginning advantage. Position remains decodable across architectures even when response curves differ, and attention concentration near initial tokens covaries with beginning sensitivity in the tested Transformer family. These exploratory results show that effective context use depends on the evaluated architecture family, capability, task, and prompt, without isolating the sequence mixer as the sole cause. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.31135 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-25T08:31:13.225Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).