Metraipolis › archive › dc_60bf314ad08b
In-Context Binding Capacity in Language Models — arXiv:2609.30634
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30634
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:10:29.683Z
- SHA-256 of the archived text
e8ac18f0e0ab5a9ace76a8ecec8fa544b84dadd883fc81063735e5580ce5a461- Stable URL of this page
https://metraipolis.com/documento/dc_60bf314ad08b
How to cite
In-Context Binding Capacity in Language Models — arXiv:2609.30634 — arXiv (en), https://arxiv.org/abs/2609.30634 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_60bf314ad08b (Metraipolis archive dc_60bf314ad08b, sha256 e8ac18f0e0ab5a9ace76a8ecec8fa544b84dadd883fc81063735e5580ce5a461)
BibTeX
@misc{dc60bf314ad08b,
title = {In-Context Binding Capacity in Language Models — arXiv:2609.30634},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30634},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_60bf314ad08b; sha256 e8ac18f0e0ab5a9ace76a8ecec8fa544b84dadd883fc81063735e5580ce5a461},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30634. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
How many assignments can a language model recall before it loses track of which value belongs to which entity? We measure this limit using continuous recall curves for 12 models at or below 3B parameters and a threshold sweep over 30 open models up to 12B. On the continuous curves, the load at which recall falls halfway to chance follows $K_{50}=cN^{\alpha}$, with $\alpha=0.820$ and $R^2=0.73$. The broader sweep shows an eightfold range associated with pretraining recipe, although the continuous curves show no detectable recipe effect after controlling for scale, with few modern models in the fit. We derive why interference can lower measured capacity by reducing single-binding recall even when the load-dependent recall profile is unchanged. Direct task training also exceeds the extrapolated zero-shot law, but different measurement criteria prevent interpreting that comparison as a capacity gain. Its formation times follow a power-law form in two independent codebases, conditional on runs that succeed. Together, these results characterize capacity at the model's query interface. Bounds on joint recall and a decomposition of policy errors connect this measurement to working memory and instruction following, without treating recall as a measure of alignment. The controlled task also provides a baseline for testing whether binding limits constrain world-state tracking; the present experiments do not measure state updates or downstream transfer.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.30634 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.687Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).