Metraipolis › archive › dc_ed8dbbaa277f
MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference — arXiv:2609.34077
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34077
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:02:05.323Z
- SHA-256 of the archived text
85372346d97d43b1bbf806809d87aaba8d3682955126df580e228b7635a1faba- Stable URL of this page
https://metraipolis.com/documento/dc_ed8dbbaa277f
How to cite
MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference — arXiv:2609.34077 — arXiv (en), https://arxiv.org/abs/2609.34077 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ed8dbbaa277f (Metraipolis archive dc_ed8dbbaa277f, sha256 85372346d97d43b1bbf806809d87aaba8d3682955126df580e228b7635a1faba)
BibTeX
@misc{dced8dbbaa277f,
title = {MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference — arXiv:2609.34077},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34077},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ed8dbbaa277f; sha256 85372346d97d43b1bbf806809d87aaba8d3682955126df580e228b7635a1faba},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34077. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-experts (MoE) language models often exceed the memory of a single GPU. Expert offloading keeps most experts in host memory and loads them on demand, so decoding speed depends on how many experts each token must fetch. Caching and prefetching reduce this cost only as far as the routing allows. Router-only fine-tuning can reshape the routing to reuse experts, but it keeps the experts frozen, so they cannot adapt to the tokens the new routing sends them. We propose MaskCoFT, a masked co-adaptive fine-tuning method that trains routers and experts together with the cross-entropy loss alone. During fine-tuning, a learnable binary mask restricts the Top-K routing of each layer to a subset of experts, and the experts adapt to the tokens redirected to them. At inference, the learned mask becomes a soft prior that re-ranks experts, so every expert remains selectable. We simulate a GPU cache of 4 experts per layer for Mixtral-8x7B and 12 for DeepSeek-V2-Lite. MaskCoFT cuts expert fetches per token by 23.7% and 10.1% relative to the base model. In real offloading system serving, it lowers the time per output token by up to 16.4% and 5.5%, respectively. Its average accuracy over nine benchmarks stays above the base model by 0.92 and 0.53 points. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.34077 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.843Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).