Metraipolis › archive › dc_551ac784d8ae
Landau theory of quenched criticality in linear in-context learning — arXiv:2608.28059
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.28059
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T11:06:46.269Z
- SHA-256 of the archived text
76742a5d8554070e74fb65820aff6fd51f2754089bf08c5091e7bf074af26495- Stable URL of this page
https://metraipolis.com/documento/dc_551ac784d8ae
How to cite
Landau theory of quenched criticality in linear in-context learning — arXiv:2608.28059 — arXiv (en), https://arxiv.org/abs/2608.28059 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_551ac784d8ae (Metraipolis archive dc_551ac784d8ae, sha256 76742a5d8554070e74fb65820aff6fd51f2754089bf08c5091e7bf074af26495)
BibTeX
@misc{dc551ac784d8ae,
title = {Landau theory of quenched criticality in linear in-context learning — arXiv:2608.28059},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.28059},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_551ac784d8ae; sha256 76742a5d8554070e74fb65820aff6fd51f2754089bf08c5091e7bf074af26495},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28059. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In-context learning (ICL) allows a pretrained model to infer a new task from examples supplied in its prompt without updating its parameters. In linear models of ICL, the prediction error develops a double-descent singularity when the number of pretraining samples becomes comparable to the number of learnable parameters. We formulate this interpolation singularity as a critical phenomenon of a quenched disordered system. By comparing annealed and quenched descriptions of the same linear ICL model, we identify the connected sample-to-sample fluctuations of the learned parameters as the microscopic origin of the singular error. A Landau potential is constructed by integrating the cavity self-consistency equation for the renormalized ridge parameter $\xi$. The role of (magnetization) order parameter is played by $\xi$, while the bare ridge parameter $\lambda$ becomes its conjugate magnetic field. The normalized sample complexity $\tau$ acts as a temperature and the double-descent singularity occurs at the critical temperature $\tau_c =1$. The Landau susceptibility is precisely the quantity that diverges in the fluctuation contribution to the prediction error. The order parameter is closely related to the fraction of zero eigenvalues of the empirical relaxation matrix in the ridgeless limit, which define flat directions in the learning dynamics. The Landau theory is generically cubic in the order parameter with critical exponents $(\beta_{\rm cr},\delta_{\rm cr},\gamma_{\rm cr})=(1,2,1)$. In the large-context regime, there appears a pseudogap-like regime characterized by suppressed order parameter. Predictions of the Landau theory are independently confirmed from numerical solutions of the original learning problem with good quantitative agreement. Our results pave the way for solid statistical-physics understanding of the interpolation criticality in linear in-context learning.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.LG).
Categorie arXiv: Disordered Systems and Neural Networks (cond-mat.dis-nn) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.28059 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T10:50:25.843Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).