Metraipolis › archive › dc_72bc3ffdf729

UNREAL: Unifying Retrieval and Long-Context with a Single Model — arXiv:2610.08463

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.08463
License (as archived): CC0-1.0
Archived: 2026-10-07T11:23:23.896Z
SHA-256 of the archived text: a1f051796626e0b3ef30069d2998dc836679deb39bb9b3b9441ee52dd551f618
Stable URL of this page: https://metraipolis.com/documento/dc_72bc3ffdf729

How to cite

UNREAL: Unifying Retrieval and Long-Context with a Single Model — arXiv:2610.08463 — arXiv (en), https://arxiv.org/abs/2610.08463 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_72bc3ffdf729 (Metraipolis archive dc_72bc3ffdf729, sha256 a1f051796626e0b3ef30069d2998dc836679deb39bb9b3b9441ee52dd551f618)

BibTeX

@misc{dc72bc3ffdf729,
  title = {UNREAL: Unifying Retrieval and Long-Context with a Single Model — arXiv:2610.08463},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08463},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_72bc3ffdf729; sha256 a1f051796626e0b3ef30069d2998dc836679deb39bb9b3b9441ee52dd551f618},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08463. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether a single model-internal mechanism can select evidence across this range. We introduce UNifying REtrieval And Long-Context with a Single Model (UNREAL), a model-native evidence selection framework to span corpus retrieval and long-context inference. UNREAL encodes chunks and derives retrieval queries directly from the frozen LLM's internal representations. It adds fewer than 500K trainable parameters and leaves the backbone unchanged. On a 3B-token, 21M-chunk Wikipedia index, all four dense and hybrid UNREAL backbones outperform state-of-the-art retriever-reranker systems. The best model raises recall from 49.1% to 73.2% on HotpotQA, from 31.7% to 60.1% on 2WikiMultiHopQA, and from 8.8% to 14.4% on MuSiQue. Applied to long-context tasks, the same selection mechanism removes distractors before generation, raising NoLiMa accuracy from 1.0% to 24.83% at its maximum context length of 128K tokens, and LV-Eval's F1 score from 49.97% to 54.66% at 256K. UNREAL also reduces FLOPs and time-to-first-token relative to full-context inference from roughly 32K tokens onward, with larger gains as context grows. Together, these results establish model-internal evidence selection as a common foundation for corpus retrieval and evidence-sparse long-context inference.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.08463 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:39.454Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).