Metraipolis › archive › dc_ef6ab979f158

Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysis — arXiv:2609.39464

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39464
License (as archived)
CC0-1.0
Archived
2026-10-07T13:46:14.208Z
SHA-256 of the archived text
c8f732415502d167ea91fa924128472811fc5f18d4df5702373f069ff7870bf3
Stable URL of this page
https://metraipolis.com/documento/dc_ef6ab979f158

How to cite

Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysis — arXiv:2609.39464 — arXiv (en), https://arxiv.org/abs/2609.39464 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef6ab979f158 (Metraipolis archive dc_ef6ab979f158, sha256 c8f732415502d167ea91fa924128472811fc5f18d4df5702373f069ff7870bf3)
BibTeX
@misc{dcef6ab979f158,
  title = {Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysis — arXiv:2609.39464},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39464},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef6ab979f158; sha256 c8f732415502d167ea91fa924128472811fc5f18d4df5702373f069ff7870bf3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39464. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In federated averaging, local objectives can admit multiple optimal heads, making the aggregate depend on which heads clients return. We study this ambiguity in federated multivariate regression with private backbones and a shared linear head, using an unconstrained feature model (UFM) that treats training-sample features as free variables. We introduce a natural selection rule: each client returns the optimal head closest to the broadcast head. We show that global minimization with a vanishing proximal penalty on the head realizes this rule. When the clients' optimal Gram matrices and the initial shared Gram matrix are positive definite, the shared Gram matrix follows a closed recursion and converges to the unique Bures-Wasserstein barycenter of the clients' optimal Gram matrices. Even with this alignment, the limit generally differs from the centralized optimal Gram matrix. We decompose this gap into three positive-semidefinite terms arising from differences in client target means, covariance heterogeneity, and averaging the aligned heads. A correction based on a one-time exchange of target means and covariances recovers the centralized optimal Gram matrix in one round under exact local optimization and the same selection rule. We verify these results numerically in the UFM and test its predictions on five tabular and five image regression datasets using deep networks with feature regularization and long local training. In these experiments, ordinary training approaches the predicted barycenter, while a weak proximal penalty improves endpoint agreement and yields trajectories that closely follow the predicted Gram dynamics. The correction moves the final Gram matrices close to the centralized UFM prediction.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.39464 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T13:18:10.390Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).