Metraipolis › archive › dc_13d37e397a3f
Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection — arXiv:2609.37842
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.37842
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:27:47.270Z
- SHA-256 of the archived text
5f8a49b21cdbdf1ed017186d5b63cdaae2efd380446b4c4efbbdffeda4042806- Stable URL of this page
https://metraipolis.com/documento/dc_13d37e397a3f
How to cite
Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection — arXiv:2609.37842 — arXiv (en), https://arxiv.org/abs/2609.37842 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_13d37e397a3f (Metraipolis archive dc_13d37e397a3f, sha256 5f8a49b21cdbdf1ed017186d5b63cdaae2efd380446b4c4efbbdffeda4042806)
BibTeX
@misc{dc13d37e397a3f,
title = {Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection — arXiv:2609.37842},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.37842},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_13d37e397a3f; sha256 5f8a49b21cdbdf1ed017186d5b63cdaae2efd380446b4c4efbbdffeda4042806},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37842. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Influence functions estimate how individual training examples affect the behavior of large language models (LLMs). Analyzing how training data influence different behaviors of an LLM involves repeated influence computation. Reusing stored training gradients reduces the computational cost, but storing full gradients is prohibitively expensive at LLM scale. We study how to compress these gradients while preserving influence estimates for future queries that are unknown at storage time. Through a worst-case analysis, we characterize the optimal fixed-dimensional linear representation and propose eigenbasis-corrected one-bit gradient projection (EOGP) to approximate it at scale. Specifically, EOGP uses EK-FAC to reduce gradient dimensionality, then applies PCA within the retained subspace to learn compression directions from the training gradients. We then apply one-bit quantization to the resulting coordinates, allowing more coordinates to be retained within a fixed storage budget. On GPT-2, EOGP predicts retraining outcomes more accurately than the evaluated compression baselines while using one-sixteenth of their per-example storage. On OLMo 2 SFT models from 1B to 32B parameters, EOGP remains competitive with the baselines allocated over 100 times as much storage per example. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.37842 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:54.036Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).