Metraipolis › archive › dc_ab75ec8d6cdd

Who Pays for the Knowledge Models Copy? Model Collapse as Rogers' Paradox — arXiv:2611.06022

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.06022
License (as archived)
CC0-1.0
Archived
2026-11-05T18:18:57.861Z
SHA-256 of the archived text
fffcb96dda369842c8021130b15b854b02ecc536397cbde1d12686c6164ad703
Stable URL of this page
https://metraipolis.com/documento/dc_ab75ec8d6cdd

How to cite

Who Pays for the Knowledge Models Copy? Model Collapse as Rogers' Paradox — arXiv:2611.06022 — arXiv (en), https://arxiv.org/abs/2611.06022 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab75ec8d6cdd (Metraipolis archive dc_ab75ec8d6cdd, sha256 fffcb96dda369842c8021130b15b854b02ecc536397cbde1d12686c6164ad703)
BibTeX
@misc{dcab75ec8d6cdd,
  title = {Who Pays for the Knowledge Models Copy? Model Collapse as Rogers' Paradox — arXiv:2611.06022},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.06022},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab75ec8d6cdd; sha256 fffcb96dda369842c8021130b15b854b02ecc536397cbde1d12686c6164ad703},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06022. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative models trained on the output of earlier models lose knowledge across generations, rare knowledge first. The remedies that work all bring in information from outside the loop, and the theory of model collapse treats that supply as given: a fixed archive, drawn from a fixed distribution. We argue that the supply is the problem, and that cultural evolution has the theory of it. We read model collapse as Rogers' paradox: a population that comes to rely on copying ends up no better off than one that never copied, because copies go stale unless someone keeps learning for themselves. We test the identification in three steps. First, Rogers' model, fitted only to fully synthetic training, predicts to within a few percentage points how much rare knowledge language models, from word trigrams to a pretrained transformer, retain under other mixtures of fresh data. Second, when the world itself changes, a lineage that copies its own past needs more fresh data to keep up the faster the world changes, as Rogers' model says it must and as analyses that assume a fixed target cannot; a lineage that restarts every generation from fixed pretrained weights is largely insulated, because those weights are knowledge that is never copied. Third, we close the loop: when the people who write a model's training data can ask the model instead of finding things out, the supply of fresh data becomes endogenous. At equilibrium, the model is exactly as reliable as it is costly for people to find out for themselves, the population is no better off than without the model, and every increase in the share of model-written text must be paid for by more people learning for themselves. Users who check what they are told and relearn when it is wrong break the paradox. Filters that select plausible text are conformist and accelerate collapse; only checks that carry new information act as critical learning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL); Populations and Evolution (q-bio.PE).
Pagina del lavoro: https://arxiv.org/abs/2611.06022 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:13:03.954Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).