Metraipolis › archive › dc_08ff5cbb4540

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation — arXiv:2608.03620

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.03620
License (as archived)
CC0-1.0
Archived
2026-10-31T12:19:29.695Z
SHA-256 of the archived text
1eb543b1e96868a3a1cc4d27c4b8c386d9983d4ba70ad5e80f516a8c9b1af400
Stable URL of this page
https://metraipolis.com/documento/dc_08ff5cbb4540

How to cite

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation — arXiv:2608.03620 — arXiv (en), https://arxiv.org/abs/2608.03620 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_08ff5cbb4540 (Metraipolis archive dc_08ff5cbb4540, sha256 1eb543b1e96868a3a1cc4d27c4b8c386d9983d4ba70ad5e80f516a8c9b1af400)
BibTeX
@misc{dc08ff5cbb4540,
  title = {A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation — arXiv:2608.03620},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.03620},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_08ff5cbb4540; sha256 1eb543b1e96868a3a1cc4d27c4b8c386d9983d4ba70ad5e80f516a8c9b1af400},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03620. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Activation patching and weight-space ablation both claim a component is causally responsible for a behavior, yet they act on different objects: one forward pass versus the parameters behind every forward pass. We ask when they agree. We study an idealized model where a conditional computation is carried additively through a residual stream, $F(x)=F_0(x)+\sum_i\alpha_i(x)v_i$, read out by a linear functional, and prove three exact results. First, deleting a subset of carriers collapses a matched input pair onto the same unconditional output \emph{if and only if} the removal is symmetric on the pair and leaves no outside contrast; the error is deterministic, and we give its exact form even when the two conditions hold only approximately. Second, patching a carrier moves the readout by its donor-receiver \emph{contrast}, while ablating it moves the readout by its \emph{absolute level}; neither bounds the other, and we construct pairs where every single-carrier patch flips the decision while no single-carrier ablation does. Third, for an attention head composed with its own layer's normalization and MLP, we derive an exact first-order interaction formula with a provably second-order remainder, vanishing identically when only the MLP is ablated but not, in general, when a head is. Small transformers trained on a synthetic conditional task illustrate all three predictions: across thirty-nine ablation configurations the measured interaction is strongly rank-correlated with the idealized model's predictive accuracy (Spearman $-0.83$), and a second task and architecture reproduces the same pattern, including a further polarity reversal. The single-block interaction result extends past one residual block, and the synthetic validation is tested against a real pretrained model, in a companion paper that takes this theory further along both axes.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.03620 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:05:53.066Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).