Metraipolis › archive › dc_2d346c6b4902

MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.26217
License (as archived)
CC0-1.0
Archived
2026-10-08T15:07:51.488Z
SHA-256 of the archived text
eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630
Stable URL of this page
https://metraipolis.com/documento/dc_2d346c6b4902

How to cite

MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217 — arXiv (en), https://arxiv.org/abs/2609.26217 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d346c6b4902 (Metraipolis archive dc_2d346c6b4902, sha256 eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630)
BibTeX
@misc{dc2d346c6b4902,
  title = {MSA-CITE: A Co-Adapted LoRA Specialist Ecology for Fixed-Budget Small-Model Inference — arXiv:2609.26217},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26217},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d346c6b4902; sha256 eaca2b9a5c375c7bfe79fad942b858ffd1df3f8859d6a3e1ce900a3fb9d6a630},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26217. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Compact language models are typically deployed by retaining a single post-training checkpoint and sampling it repeatedly. In this work, we challenge this practice by treating multiple discarded checkpoints as composable assets for deployment. Starting from a single Qwen3-4B backbone, we preserve four frozen LoRA branches, each derived from a different post-training trajectory. stead of drawing four generations from one branch, we allocate a fixed four-generation budget by sampling one completion from each branch. Our method, Multi-path Specialist Adaptation with Calibrated Inference-Time Evidence (MSA-CITE), processes the resulting portfolio by grouping terminal answers into equivalence classes, scoring each class via summed calibration-derived source priors, and selecting a representative under deterministic tie-breaking rules. The readout stage does not learn from evaluation results, nor does it introduce additional generations, verifiers, or reranking steps. On 200 held-out mathematics items, the four-path portfolio achieves 65.5% accuracy, compared with 62.0% for the strongest single-branch baseline. On a 100-item subject-disjoint shift, it attains 42.0% versus 40.0%. Under in-distribution conditions, the improvements over homogeneous SFT and Online-OPD repetition are robust; results against the strongest baseline and under shifted conditions are not conclusive. Our findings offer a narrow but concrete contribution: post-training branches, even without co-training, can be collectively beneficial for deployment.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.26217 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T14:37:29.470Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).