Metraipolis › archive › dc_f91509beaebe
Mapping and Measuring the Behavioral Evolution of Large Language Models — arXiv:2608.11027
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11027
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T06:14:32.188Z
- SHA-256 of the archived text
d7fe8533caec042ceb2796d4d8056198609bb26303fa4d6752c0d98318a00fd7- Stable URL of this page
https://metraipolis.com/documento/dc_f91509beaebe
How to cite
Mapping and Measuring the Behavioral Evolution of Large Language Models — arXiv:2608.11027 — arXiv (en), https://arxiv.org/abs/2608.11027 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f91509beaebe (Metraipolis archive dc_f91509beaebe, sha256 d7fe8533caec042ceb2796d4d8056198609bb26303fa4d6752c0d98318a00fd7)
BibTeX
@misc{dcf91509beaebe,
title = {Mapping and Measuring the Behavioral Evolution of Large Language Models — arXiv:2608.11027},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11027},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f91509beaebe; sha256 d7fe8533caec042ceb2796d4d8056198609bb26303fa4d6752c0d98318a00fd7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11027. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations. We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10{,}000 prompts. After embedding each response, we construct three complementary sentence-level dissimilarities: an aligned mean per-prompt distance, which is a pseudometric on observed model responses; a PCA-compressed summary of prompt-wise disagreement; and an alignment-free Gromov--Wasserstein discrepancy between models' internal response geometries. We use these constructions to study static organization and temporal change on a release-date axis through behavioral maps, family-wise drift, hierarchical clustering, cross-family convergence, and response-cloud dispersion. Across the three constructions, model families form coherent clusters, with \texttt{gpt-2} as a global outlier; cross-family distances decrease over time; and several recent reasoning-oriented models have comparatively compact response clouds. A token-level cross-check based on per-prompt Maximum Mean Discrepancy closely agrees with the sentence-level mean distance (Spearman $\rho=0.98$) and recovers the same qualitative findings. We organize these comparisons through a measure-theoretic lens making their alignment and invariance assumptions explicit. We also establish an architecture-agnostic sufficient condition linking behavioral similarity to inference-prompt coverage, small excess population log-loss, and similar effective target distributions---a possible training-side account rather than an empirical explanation of the observed trends. Our pipeline is label-free, and re-encoding every response with three further encoders---down to one $73\times$ smaller---preserves the rank geometry, the outliers, and the sign of the time trend.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.11027 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:08:50.370Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).