Metraipolis › archive › dc_84673d249f61

ASCENT: Scale-Complementary External State for Frozen Long-Context Language Models — arXiv:2611.10175

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.10175
License (as archived): CC0-1.0
Archived: 2026-11-10T18:44:23.616Z
SHA-256 of the archived text: 9132e50e772b3f6742dd7e93c162f2bf4db568ba1d5bf691c28da1b30bca911d
Stable URL of this page: https://metraipolis.com/documento/dc_84673d249f61

How to cite

ASCENT: Scale-Complementary External State for Frozen Long-Context Language Models — arXiv:2611.10175 — arXiv (en), https://arxiv.org/abs/2611.10175 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84673d249f61 (Metraipolis archive dc_84673d249f61, sha256 9132e50e772b3f6742dd7e93c162f2bf4db568ba1d5bf691c28da1b30bca911d)

BibTeX

@misc{dc84673d249f61,
  title = {ASCENT: Scale-Complementary External State for Frozen Long-Context Language Models — arXiv:2611.10175},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10175},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84673d249f61; sha256 9132e50e772b3f6742dd7e93c162f2bf4db568ba1d5bf691c28da1b30bca911d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10175. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

External state should become more valuable as a reader becomes more capable, yet long-context evaluations measure memory utility at a single model scale. This paper formulates scale complementarity: the absolute gain from external state increases when reader capacity and a compact state budget are jointly scaled. ASCENT realizes this principle with a target-blind causal writer, nested typed state, and a bounded query-conditioned readout; every language model remains frozen. Under log loss, the value of a nested refinement equals its target-relevant conditional information; a full factorial makes the reader-by-state interaction identifiable. Official 16K BABILong experiments show Qwen2.5 accuracy gains of 10.50, 78.88, and 82.63 percentage points at 0.5B, 1.5B, and 3B parameters, with both adjacent directional contrasts significant after family-wise correction. At 7B, Foundation accuracy rises to 33.25% while four-slot ASCENT answers every question, which caps the gain at 66.75 points; the 3B-to-7B accuracy increment is -15.88 points [-20.55, -11.20]. The accuracy-side claim is therefore established from 0.5B to 3B; the 7B conclusion rests on the non-saturated negative-log-likelihood factorial. A three-model by three-state factorial yields co-scaled negative log-likelihood gains of 0.388, 2.099, and 7.406 nats at 1.5B, 3B, and 7B; all interaction and gain contrasts retain positive simultaneous lower bounds. Five public 7B-8B readers obtain 66.75-85.63 point gains, and ASCENT exceeds task-trained retrieval checkpoints at the two larger shared scales. Controls, baselines, systems accounting, matched-hardware reproduction, and a cross-architecture sensitivity analysis accompany the experiments. The results establish scale-complementary external state under the evaluated benchmarks, metrics, and state schema.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.10175 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:52.420Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).