Metraipolis › archive › dc_d4ef41d747f7

No More Free Lunch: Corpus Task Complexity Matters as Corpora Grow — arXiv:2609.29245

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29245
License (as archived)
CC0-1.0
Archived
2026-10-08T04:27:23.070Z
SHA-256 of the archived text
49fcfb19ab5628794dfd5ee2fcee490e6b053cf8e564d7b60f5ba5c627b85df0
Stable URL of this page
https://metraipolis.com/documento/dc_d4ef41d747f7

How to cite

No More Free Lunch: Corpus Task Complexity Matters as Corpora Grow — arXiv:2609.29245 — arXiv (en), https://arxiv.org/abs/2609.29245 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4ef41d747f7 (Metraipolis archive dc_d4ef41d747f7, sha256 49fcfb19ab5628794dfd5ee2fcee490e6b053cf8e564d7b60f5ba5c627b85df0)
BibTeX
@misc{dcd4ef41d747f7,
  title = {No More Free Lunch: Corpus Task Complexity Matters as Corpora Grow — arXiv:2609.29245},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29245},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4ef41d747f7; sha256 49fcfb19ab5628794dfd5ee2fcee490e6b053cf8e564d7b60f5ba5c627b85df0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29245. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Given a large corpus, the questions one might ask can vary -- from "When was the first human heart transplant?" to "What are all the contradictory claims in this literature?" -- but what makes some questions more challenging than others? In this work, we define a notion of Corpus Task Complexity (CTC) that characterizes tasks by how their difficulty grows with corpus size; for instance, a retrieval query only requires a single linear pass over a corpus, while finding contradictions requires checking a quadratically growing set of claim pairs. Observing that prior work has largely only studied tasks whose difficulty grows linearly with corpus size, which we call low CTC tasks, we introduce 10 new tasks belonging to a class of high CTC whose difficulty grows quadratically or more in corpus size. We find that high-CTC tasks not only grow much more challenging on average at longer contexts for LCLMs, they reverse many modeling conclusions drawn solely from low-CTC evaluations. For instance, efficient block-sparse and hybrid attention approaches consistently match full attention performance on low-CTC tasks, but degrade much more on high-CTC tasks. Large-corpus high-CTC reasoning thus remains an open challenge as full attention is too costly to scale, motivating future research on these tasks. We release our code, data, and 22-task suite (CTC-Bench), to facilitate future research in this area.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.29245 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.030Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).