Metraipolis › archive › dc_69bf522297b2
Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation) — arXiv:2608.14563
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14563
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:47:27.493Z
- SHA-256 of the archived text
224e0733a336c625385c04e09adcba06987eeb42b64e934ef13e1c1600466e49- Stable URL of this page
https://metraipolis.com/documento/dc_69bf522297b2
How to cite
Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation) — arXiv:2608.14563 — arXiv (en), https://arxiv.org/abs/2608.14563 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_69bf522297b2 (Metraipolis archive dc_69bf522297b2, sha256 224e0733a336c625385c04e09adcba06987eeb42b64e934ef13e1c1600466e49)
BibTeX
@misc{dc69bf522297b2,
title = {Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation) — arXiv:2608.14563},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14563},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_69bf522297b2; sha256 224e0733a336c625385c04e09adcba06987eeb42b64e934ef13e1c1600466e49},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14563. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Forward-Pass-Only MLP training (FPO) adapts large language models without a backward pass through the model body, achieving 2.7--3.2x the throughput of standard fine-tuning at ~40% less peak training memory, while leaving off-domain benchmarks within seed-noise of baseline, a property that full-network fine-tuning does not reliably reproduce. FPO rests on a single empirical observation: at late layers of a transformer, the output-layer prediction error approximates the true gradient with cosine similarity 0.47--0.59 across six public models we survey. We introduce a two-minute diagnostic that quantifies this approximation per layer for any model, identifying where late-layer adaptation is viable. Informed by the diagnostic, FPO computes a single error signal at the output and applies it to each target layer. No signal is propagated between layers, and no autograd graph is constructed at any point. We evaluate FPO on three model families (OLMo-2-7B, Qwen3-8B, Falcon3-7B). Across all three, FPO produces in-domain perplexity improvement and leaves MMLU, ARC-Challenge, HellaSwag, and Winogrande within seed-noise of baseline. Localizing SFT to FPO's target layers to enter this regime is also feasible, but at 2.2x the wall-clock cost of FPO. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14563 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.898Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).