Metraipolis › archive › dc_d57fb06ddb49

When the Model Retires: An Empirical Study of LLM Migration in Open-Source Applications — arXiv:2609.31288

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31288
License (as archived)
CC0-1.0
Archived
2026-10-08T02:45:10.170Z
SHA-256 of the archived text
439529ad2d14fc509659ba3607730913f75d94c8ceaef435ca868f98d26b2f05
Stable URL of this page
https://metraipolis.com/documento/dc_d57fb06ddb49

How to cite

When the Model Retires: An Empirical Study of LLM Migration in Open-Source Applications — arXiv:2609.31288 — arXiv (en), https://arxiv.org/abs/2609.31288 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d57fb06ddb49 (Metraipolis archive dc_d57fb06ddb49, sha256 439529ad2d14fc509659ba3607730913f75d94c8ceaef435ca868f98d26b2f05)
BibTeX
@misc{dcd57fb06ddb49,
  title = {When the Model Retires: An Empirical Study of LLM Migration in Open-Source Applications — arXiv:2609.31288},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31288},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d57fb06ddb49; sha256 439529ad2d14fc509659ba3607730913f75d94c8ceaef435ca868f98d26b2f05},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31288. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Applications built on commercial large language model (LLM) APIs depend on model versions that providers retire on their own schedule, with notice periods ranging from one year to two weeks. We ask what actually happens to applications when a model is retired. We mine GitHub for commits that migrate away from officially deprecated models and endpoints of OpenAI, Anthropic, and Google, matching each commit to the provider's published announcement and shutdown dates. From 22,555 commits in 17,703 non-fork repositories (2024-2026), 5,139 are matched to an official event; two independent coders validated a stratified sample of 300 (kappa = 0.89-0.95), and we reweight all estimates by their labels. We find that an estimated 82% (95% CI 79-84) of migrations away from retired models were committed after the shutdown date - after the application had started failing - regardless of repository popularity, prior retirement experience, or the presence of a provider-abstraction layer. The share tracks the provider's notice policy: 89% for Anthropic's 60-114-day notices versus 13% for OpenAI's one-year Assistants API notice, and each e-fold increase in notice length reduces the odds of post-shutdown migration by about three quarters. Model identifiers are hard-coded in 94% of migrating applications, migration effort scales from a median of 6 added lines for prompt-only applications to nearly 700 for fine-tuned ones, and only 8% of migrations switch provider. We release the dataset and pipeline and discuss implications for deprecation policy, dependency-risk assessment of LLM products, and tooling.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-28 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.31288 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T02:09:54.647Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).