Metraipolis › archive › dc_cc8806e0bdcb

What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment — arXiv:2609.32485

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32485
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:37.458Z
SHA-256 of the archived text
7a87854d7dbc8c2d5430d5b8e27410adc81f80c64e6c4011c1eaa581c937fc71
Stable URL of this page
https://metraipolis.com/documento/dc_cc8806e0bdcb

How to cite

What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment — arXiv:2609.32485 — arXiv (en), https://arxiv.org/abs/2609.32485 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cc8806e0bdcb (Metraipolis archive dc_cc8806e0bdcb, sha256 7a87854d7dbc8c2d5430d5b8e27410adc81f80c64e6c4011c1eaa581c937fc71)
BibTeX
@misc{dccc8806e0bdcb,
  title = {What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment — arXiv:2609.32485},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32485},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cc8806e0bdcb; sha256 7a87854d7dbc8c2d5430d5b8e27410adc81f80c64e6c4011c1eaa581c937fc71},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32485. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Federated low-rank adaptation (LoRA) requires identifying an update structure that is shared across heterogeneous clients. Prior work reports strong similarity among trained LoRA projection matrices across clients; however, such agreement may be largely induced by common initialization and collapses toward random overlap under independent initialization. More crucially, relying solely on parameter similarity inherently ignores the influence of local input regime. To uncover a more robust shared structure, we introduce an input-aware action matrix that weights the adapter update by the second-moment statistics of local layer inputs. Empirically, while parameter similarity vanishes, the leading right singular directions of this action matrix remain strongly aligned across clients. This shared geometry preserves task-conditioned differences and naturally varies across network depths. Motivated by these findings, we propose Federated Subspace-Guided Action-Informed Learning (FedSAIL). Instead of averaging weights, FedSAIL estimates a shared action subspace to regularize local training while preserving client-specific coefficients. Across several benchmarks, our approach consistently improves predictive performance over competing federated LoRA methods while reducing communication cost significantly.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32485 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.696Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).