Metraipolis › archive › dc_52d14d607455
SynMOE: A Syntax-Aware MOE-Guided Approach for PEFT-Based Code Intelligence Learning — arXiv:2611.06069
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.06069
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:26:25.515Z
- SHA-256 of the archived text
40ca6c72e29d41893a5b976cd277c2780edf346a3de9b0576d65a5a3334dd271- Stable URL of this page
https://metraipolis.com/documento/dc_52d14d607455
How to cite
SynMOE: A Syntax-Aware MOE-Guided Approach for PEFT-Based Code Intelligence Learning — arXiv:2611.06069 — arXiv (en), https://arxiv.org/abs/2611.06069 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_52d14d607455 (Metraipolis archive dc_52d14d607455, sha256 40ca6c72e29d41893a5b976cd277c2780edf346a3de9b0576d65a5a3334dd271)
BibTeX
@misc{dc52d14d607455,
title = {SynMOE: A Syntax-Aware MOE-Guided Approach for PEFT-Based Code Intelligence Learning — arXiv:2611.06069},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.06069},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_52d14d607455; sha256 40ca6c72e29d41893a5b976cd277c2780edf346a3de9b0576d65a5a3334dd271},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06069. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Parameter-Efficient Fine-Tuning (PEFT) has emerged as an important approach for adapting Language Models (LMs) in downstream tasks. However, in the domain of code intelligence, the performance of most PEFT methods relatively drops compared to Full fine-Tuning (FT), hindering their broader applicability. One of the main reasons is that current PEFT methods are generally based on assumptions about the low-rank nature or sparsity of parameter updates, lacking explicit modeling of code-related information. To tackle this limitation, we propose SynMOE, a syntax-aware MOE-guided approach for PEFT on code intelligence learning. Specifically, SynMOE contains two stages: syntactic information extraction and syntax-aware MOE-guided adaptation. It first obtains the syntactic labels of source code from Abstract Syntax Trees (ASTs), and then aligns the labels with the input tokens of code LMs. In the second stage, syntactic labels are integrated into the MOE module, participating in the routing decision of MOE for a more powerful syntactic-aware token representation. We evaluate the effectiveness of the proposed approach by applying it to two typical PEFT methods, LoRA and $(IA)^3$, across three diverse code intelligence tasks. Experimental results show that SynMOE achieves consistent and significant improvements. Even compared to FT, SynMOE achieves comparable or superior performance while being more computationally efficient. In addition, we also conduct ablation studies, hyperparameter analysis, and evaluation in the low-resource scenario to provide a comprehensive investigation of the proposed approach. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2611.06069 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:16:03.881Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).