Metraipolis › archive › dc_1cc10cbf23d2

Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory — arXiv:2608.09997

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09997
License (as archived)
CC0-1.0
Archived
2026-10-19T06:19:12.848Z
SHA-256 of the archived text
0dd398b991be3e7e92926d2711b273f5420f1bb38c002a1cbf7bb7263e92fbec
Stable URL of this page
https://metraipolis.com/documento/dc_1cc10cbf23d2

How to cite

Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory — arXiv:2608.09997 — arXiv (en), https://arxiv.org/abs/2608.09997 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1cc10cbf23d2 (Metraipolis archive dc_1cc10cbf23d2, sha256 0dd398b991be3e7e92926d2711b273f5420f1bb38c002a1cbf7bb7263e92fbec)
BibTeX
@misc{dc1cc10cbf23d2,
  title = {Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory — arXiv:2608.09997},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09997},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1cc10cbf23d2; sha256 0dd398b991be3e7e92926d2711b273f5420f1bb38c002a1cbf7bb7263e92fbec},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09997. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Transformers have had a profound impact on the world of language processing and computer vision. As efforts to answer the million-dollar question of ``How does a Transformer learn?" have been increasing, existing interpretability studies primarily analyze representations at isolated layers or the network as a whole, while the developmental evolution of individual representations and its manifolds across transformer layers remains underexplored. With this work, we aim at providing a comprehensive analysis of the evolution of representations as the representation point cloud transforms across the layers; thereby attempting to isolate layers or establish a trend which comes closer to justifying how and when raw input representations evolve into task-relevant feature representations. Thus, Transformer Geometry Observatory-TGO-IV introduces a topological framework for analysing the evolution of Transformer representations through the lens of Persistent Homology. Rather than studying local geometric properties alone, TGO-IV constructs Vietoris--Rips simplicial complexes from token-level representation point clouds and investigates the evolution of their persistent topological signatures across Transformer layers. The proposed framework comprises complementary topological observatories including Persistence Diagrams, Barcode Diagrams, Betti Curves, Persistence Landscapes, Bottleneck Distance, and Wasserstein Distance, enabling a comprehensive analysis of how the global topology of representation point clouds develops throughout the forward pass.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.09997 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:11:45.059Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).