Metraipolis › archive › dc_57a429fe3532
Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates — arXiv:2611.04797
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04797
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T18:54:13.516Z
- SHA-256 of the archived text
9fef689baf9d9ad81f227ae71501e6b24356e47ab41904ab545d608ba8f1c0d3- Stable URL of this page
https://metraipolis.com/documento/dc_57a429fe3532
How to cite
Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates — arXiv:2611.04797 — arXiv (en), https://arxiv.org/abs/2611.04797 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_57a429fe3532 (Metraipolis archive dc_57a429fe3532, sha256 9fef689baf9d9ad81f227ae71501e6b24356e47ab41904ab545d608ba8f1c0d3)
BibTeX
@misc{dc57a429fe3532,
title = {Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates — arXiv:2611.04797},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04797},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_57a429fe3532; sha256 9fef689baf9d9ad81f227ae71501e6b24356e47ab41904ab545d608ba8f1c0d3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04797. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) on personal devices keep user data local, but limited memory restricts their general capability. Since most user-specific tasks do not require the breadth of frontier models, a practical solution is to adapt a shared on-device model into lightweight specialists. However, backpropagation is prohibitive in memory, and users rarely provide labeled data. We present MobileSpec, a system that trains specialists with forward passes alone and learns from automatically checked task outcomes, as in reinforcement learning with verifiable rewards (RLVR). Because forward-only updates are built from random perturbations, training is efficient only when these perturbations resemble the weight changes of RLVR, which largely preserve the singular values of pretrained weights and rotate their singular vectors. The core algorithm of MobileSpec, FuRA-S, produces such perturbations while training only the block-spectral coordinates of pretrained weights, namely the singular values of their column blocks. As we show theoretically and empirically, rescaling these blocks independently shifts their relative contributions to each weight matrix and thereby rotates its singular vectors. Since this update can be rewritten as a transformation of the layer input, FuRA-S also adapts INT4 models without modifying their weights. With specialists of only 1-5 MB that update 0.015-0.05% of the parameters, MobileSpec raises the success rate of a phone-use agent from 6-23% up to 89-100% and trains a personalized 8B model on an iPhone 16 Pro overnight. On RLVR benchmarks, FuRA-S is competitive with full-parameter evolution strategies (ES) and outperforms ES over LoRA adapters and quantized ES on INT4 models. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Systems and Control (eess.SY). Pagina del lavoro: https://arxiv.org/abs/2611.04797 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:35:37.459Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).