Metraipolis › archive › dc_12be95d90ebc
What does FFN compression change downstream? Same-state causal restoration in diffusion language models — arXiv:2609.31685
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.31685
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:59:23.642Z
- SHA-256 of the archived text
bb41d93fceb0e6638bd09ac12b84c4f947b1dc0e9d2d64e63833ffef14ab96e0- Stable URL of this page
https://metraipolis.com/documento/dc_12be95d90ebc
How to cite
What does FFN compression change downstream? Same-state causal restoration in diffusion language models — arXiv:2609.31685 — arXiv (en), https://arxiv.org/abs/2609.31685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_12be95d90ebc (Metraipolis archive dc_12be95d90ebc, sha256 bb41d93fceb0e6638bd09ac12b84c4f947b1dc0e9d2d64e63833ffef14ab96e0)
BibTeX
@misc{dc12be95d90ebc,
title = {What does FFN compression change downstream? Same-state causal restoration in diffusion language models — arXiv:2609.31685},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.31685},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_12be95d90ebc; sha256 bb41d93fceb0e6638bd09ac12b84c4f947b1dc0e9d2d64e63833ffef14ab96e0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion language models (DLMs) enable flexible, parallel generation, but their iterative denoising remains computationally expensive, motivating increasingly aggressive compression. Existing compression objectives largely measure how well compressed computation approximates the original locally, but local error does not reveal which removed computations actually matter to the downstream denoising trajectory. We introduce Same-State Causal Restoration (SSR), which restores the original FFN on the exact current input reached by the compressed model and measures how the resulting trajectory changes. To our knowledge, this is the first direct measurement of the same-current-input closed-loop effect of removed FFN computation in DLM compression. Across LLaDA-8B-Instruct and Dream-v0-Instruct-7B, compressed-side state ranks this downstream effect substantially better than local NMSE at fixed denoising phase, while controlled interventions show that correction structure matters beyond magnitude. Using task-label-free calibration, SSR freezes a single restoration window for held-out inference. Under aggressive LLaDA compression, restoring only four transitions recovers 89.9% of the lost accuracy while retaining an estimated 36.8% whole-model MAC saving and outperforming an equal-budget local-error baseline. Dream further shows that restoring dense behavior and repairing the final task are distinct outcomes. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.31685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.029Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).