Metraipolis › archive › dc_e331a1d7bf29

The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era — arXiv:2608.28980

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.28980
License (as archived)
CC0-1.0
Archived
2026-10-17T07:47:25.766Z
SHA-256 of the archived text
371dc99041578a84b8aec06e6ea3a6d671a78480269f2d0cb5d8ece3e3eb051f
Stable URL of this page
https://metraipolis.com/documento/dc_e331a1d7bf29

How to cite

The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era — arXiv:2608.28980 — arXiv (en), https://arxiv.org/abs/2608.28980 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e331a1d7bf29 (Metraipolis archive dc_e331a1d7bf29, sha256 371dc99041578a84b8aec06e6ea3a6d671a78480269f2d0cb5d8ece3e3eb051f)
BibTeX
@misc{dce331a1d7bf29,
  title = {The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era — arXiv:2608.28980},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.28980},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e331a1d7bf29; sha256 371dc99041578a84b8aec06e6ea3a6d671a78480269f2d0cb5d8ece3e3eb051f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28980. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Specialized machine learning architectures encode structural assumptions (equivariance, permutation invariance, relational structure) that language-based foundation models lack by design. This review asks whether such assumptions can instead be acquired through language, drawing on a corpus of 186 papers published between 2016 and 2026 across nine modalities: tabular data, graphs, time series, vision, chemistry, code, knowledge graphs, point clouds, and protein structure. Methods are organized into eight representational regimes, ranging from language-only prompting to fully specialized architectures, and are assessed not only on predictive accuracy but on whether structural information is preserved by the representation and computed by the model. Language-mediated models prove competitive in extreme few-shot prediction, discretized symbolic tasks, and textually annotated knowledge graphs. No evidence of general replacement survives once structure itself is evaluated: where direct tests exist, apparent parity is traceable to information asymmetry, benchmark contamination, or computation performed outside the language model. Across independent research communities, missing structural inductive biases are reintroduced through graph modules, structure-aware tokens, or specialized attention, indicating that specialization is relocated rather than eliminated. The review closes by specifying a falsifiable controlled experiment that would measure the remaining gap directly. The derived data supporting the findings of this review are openly available in the repository at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.28980 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.708Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).