Metraipolis › archive › dc_413eb3fb13a5

When Search Eats the Web: A Model of Corpus Erosion under Generative Extraction — arXiv:2608.15896

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15896
License (as archived)
CC0-1.0
Archived
2026-10-18T19:08:03.781Z
SHA-256 of the archived text
7d87e61fa7b316c445ad931901e1f08a8a19e66fd341953db9abfc3ff2e9e6ff
Stable URL of this page
https://metraipolis.com/documento/dc_413eb3fb13a5

How to cite

When Search Eats the Web: A Model of Corpus Erosion under Generative Extraction — arXiv:2608.15896 — arXiv (en), https://arxiv.org/abs/2608.15896 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_413eb3fb13a5 (Metraipolis archive dc_413eb3fb13a5, sha256 7d87e61fa7b316c445ad931901e1f08a8a19e66fd341953db9abfc3ff2e9e6ff)
BibTeX
@misc{dc413eb3fb13a5,
  title = {When Search Eats the Web: A Model of Corpus Erosion under Generative Extraction — arXiv:2608.15896},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15896},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_413eb3fb13a5; sha256 7d87e61fa7b316c445ad931901e1f08a8a19e66fd341953db9abfc3ff2e9e6ff},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15896. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative search engines (GSEs) answer user queries directly from crawled web content. The capture of value from the corpus without a visit returned to the source (we call this capture extraction) diverts the traffic that finances content production. In response, publishers may restrict crawler access to their websites. In this paper, we model the crawlable corpus as a common-pool resource: the crawlable commons. It is described by three quantities: volume, average quality, and lifetime. Under two types of responses of publishers we prove that extraction degrades all three at once: publishers opt out, renewal loses its funding, and content becomes more perishable. After a given erosion threshold, the corpus goes extinct. A myopic GSE can cross this threshold, a long-run oriented GSE stays below it. We extend our model to several competing engines and prove, under a concavity condition on the steady-state value of the commons, that the symmetric equilibrium extraction rate is nondecreasing in their number and converges to the threshold. Adding users who strictly prefer direct answers, the assumption most favorable to extraction, we prove that the socially optimal extraction rate lies strictly below the erosion threshold, and no higher than the single engine's sustainable optimum. Finally, we discuss seven survival mechanisms.

Tema degli esploratori: economia dei servizi IA (retro-catalogo arXiv: annunci del 2026-08-18 in cs.GT).
Categorie arXiv: Computer Science and Game Theory (cs.GT) ; Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2608.15896 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T18:12:50.745Z dalla pagina https://arxiv.org/catchup/cs.GT/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).