Metraipolis › archive › dc_bd5f36dca041

NinaXander: Feasibility and Limits of Composing Frozen Language Models Across Architecture Families via a Shared Latent Space — arXiv:2609.38261

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38261
License (as archived)
CC0-1.0
Archived
2026-10-07T12:43:23.872Z
SHA-256 of the archived text
4e3622a8ed7c9a2ff9ba9ec9495a1be15db0ad3690f89003b0941045667316bc
Stable URL of this page
https://metraipolis.com/documento/dc_bd5f36dca041

How to cite

NinaXander: Feasibility and Limits of Composing Frozen Language Models Across Architecture Families via a Shared Latent Space — arXiv:2609.38261 — arXiv (en), https://arxiv.org/abs/2609.38261 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd5f36dca041 (Metraipolis archive dc_bd5f36dca041, sha256 4e3622a8ed7c9a2ff9ba9ec9495a1be15db0ad3690f89003b0941045667316bc)
BibTeX
@misc{dcbd5f36dca041,
  title = {NinaXander: Feasibility and Limits of Composing Frozen Language Models Across Architecture Families via a Shared Latent Space — arXiv:2609.38261},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38261},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd5f36dca041; sha256 4e3622a8ed7c9a2ff9ba9ec9495a1be15db0ad3690f89003b0941045667316bc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38261. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In this paper we propose NinaXander, a series of composed language models obtained by connecting layers of frozen language models from different architecture families with a single trained shared-latent adapter. A composed model runs the first layers of one model, converts the resulting intermediate representation once with the adapter, and then runs the remaining layers of the other model. Once the adapter is trained, several composed models that connect at different layers are obtained without retraining. Using the recurrent RWKV-4-Raven-7B and the Transformer-based Tulu-Pythia-6.9b, abbreviated as RWKV and Pythia, this study examines whether frozen models from different families can be recombined post hoc. The composed models answered multiple-choice questions, and those whose generations we examined produced syntactically well-formed text. The configuration that combines the first 5 layers of Pythia with the remaining 27 layers of RWKV reduced the Transformer key-value (KV) cache by 84.4% with accuracy not significantly different from that of RWKV alone. In multiple-choice accuracy, however, no composed model matched the parent model Pythia, and language-modeling performance decreased sharply on WikiText, a corpus of Wikipedia articles outside the training domain. The correspondence between intermediate representations was also obtained in one favorable case, with a shared tokenizer, the same depth, and the same hidden width, and does not show that the models share a general semantic space.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38261 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.842Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).