Metraipolis › archive › dc_ed958fbdb4f6
Adapters as Data: The Latent Space of LoRAs — arXiv:2611.07775
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.07775
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:50:44.287Z
- SHA-256 of the archived text
bb8146a2939a05993ce19481db233cc19b3770de7b6cedc0ea38d1d17ed97c2b- Stable URL of this page
https://metraipolis.com/documento/dc_ed958fbdb4f6
How to cite
Adapters as Data: The Latent Space of LoRAs — arXiv:2611.07775 — arXiv (en), https://arxiv.org/abs/2611.07775 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ed958fbdb4f6 (Metraipolis archive dc_ed958fbdb4f6, sha256 bb8146a2939a05993ce19481db233cc19b3770de7b6cedc0ea38d1d17ed97c2b)
BibTeX
@misc{dced958fbdb4f6,
title = {Adapters as Data: The Latent Space of LoRAs — arXiv:2611.07775},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.07775},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ed958fbdb4f6; sha256 bb8146a2939a05993ce19481db233cc19b3770de7b6cedc0ea38d1d17ed97c2b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07775. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LoRA adapters are usually treated as finished artifacts rather than as a data modality. Prior work has shown that LoRA adapters populate a low-dimensional manifold that a linear PCA basis already organizes well enough to support different operations such as sampling, editing, and inversion. We ask what a learned, nonlinear latent adds beyond this. We train a transformer autoencoder on the same population of LoRA identity adapters, mapping each into a latent space and compare every operation against the identical operation applied directly to the raw adapter weights. The latent preserves the capabilities already available in weight space, sampling diverse identities and editing individual attributes. Beyond preservation, we show that the latent is far more robust to displacement than weight space, degrading into distorted but still valid faces rather than texture. Fixed-dimensional weight-space representations cannot natively host adapters of more than one rank, so we extend our sequential autoencoder to hold mixed-rank adapters within one shared latent. This gives a path toward treating adapter weights of varying architecture as a single, general modality rather than one tied to a fixed rank, and points to a learned, nonlinear latent as the mechanism that could carry this generalization further, toward heterogeneous populations of adapters. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2611.07775 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:21:42.635Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).