Metraipolis › archive › dc_f536b6bed5bc
Internalizer: Portable Context-to-Parameter Mapping for Very Large Language Models — arXiv:2610.11715
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.11715
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T05:48:17.707Z
- SHA-256 of the archived text
f91f03c90bdf61a51b132c68b9e63b26d18e56f7b729fbe74f22ce29abfa22b6- Stable URL of this page
https://metraipolis.com/documento/dc_f536b6bed5bc
How to cite
Internalizer: Portable Context-to-Parameter Mapping for Very Large Language Models — arXiv:2610.11715 — arXiv (en), https://arxiv.org/abs/2610.11715 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f536b6bed5bc (Metraipolis archive dc_f536b6bed5bc, sha256 f91f03c90bdf61a51b132c68b9e63b26d18e56f7b729fbe74f22ce29abfa22b6)
BibTeX
@misc{dcf536b6bed5bc,
title = {Internalizer: Portable Context-to-Parameter Mapping for Very Large Language Models — arXiv:2610.11715},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.11715},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f536b6bed5bc; sha256 f91f03c90bdf61a51b132c68b9e63b26d18e56f7b729fbe74f22ce29abfa22b6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11715. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Hypernetworks that map a context directly to a LoRA adapter let a large language model carry that context in its weights, but prior work has demonstrated them only on base models of up to 14 billion parameters. We present the Internalizer, a state-of-the-art, portable Context-to-Parameter Mapping hypernetwork that generates document-specific LoRA adapters for the frozen 284B-parameter DeepSeek v4 Flash, a target two orders of magnitude larger than in any previous work. Most of its parameters live in a model-agnostic trunk with only thin entry and exit layers per base model, so it trains cheaply against small models before being ported to the large one. On unseen documents of up to 4096 tokens, the generated adapters reach 84.9% top-1 and 97.8% top-5 teacher-forced accuracy against 63.4% and 83.5% for the base model, with nothing in the context window but a three-word instruction. Once the hypernetwork is trained, a single forward pass turns any document into an adapter for such a model, which could be served alone for speed or alongside the document in the window to raise accuracy further. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.11715 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:00.899Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).