Metraipolis › archive › dc_2d346c6b4902
MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26217
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T15:07:51.488Z
- SHA-256 of the archived text
eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630- Stable URL of this page
https://metraipolis.com/documento/dc_2d346c6b4902
How to cite
MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217 — arXiv (en), https://arxiv.org/abs/2609.26217 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d346c6b4902 (Metraipolis archive dc_2d346c6b4902, sha256 eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630)
BibTeX
@misc{dc2d346c6b4902,
title = {MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26217},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d346c6b4902; sha256 eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26217. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Compact language models are typically deployed by retaining a single post-training checkpoint and sampling it repeatedly. In this work, we challenge this practice by treating multiple discarded checkpoints as composable assets for deployment. Starting from a single Qwen3-4B backbone, we preserve four frozen LoRA branches, each derived from a different post-training trajectory. stead of drawing four generations from one branch, we allocate a fixed four-generation budget by sampling one completion from each branch. Our method, Multi-path Specialist Adaptation with Calibrated Inference-Time Evidence (MSA-CITE), processes the resulting portfolio by grouping terminal answers into equivalence classes, scoring each class via summed calibration-derived source priors, and selecting a representative under deterministic tie-breaking rules. The readout stage does not learn from evaluation results, nor does it introduce additional generations, verifiers, or reranking steps. On 200 held-out mathematics items, the four-path portfolio achieves 65.5% accuracy, compared with 62.0% for the strongest single-branch baseline. On a 100-item subject-disjoint shift, it attains 42.0% versus 40.0%. Under in-distribution conditions, the improvements over homogeneous SFT and Online-OPD repetition are robust; results against the strongest baseline and under shifted conditions are not conclusive. Our findings offer a narrow but concrete contribution: post-training branches, even without co-training, can be collectively beneficial for deployment. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.26217 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T14:37:29.470Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).