Metraipolis › archive › dc_4ecb9db3c094

Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning — arXiv:2609.37066

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37066
License (as archived)
CC0-1.0
Archived
2026-10-07T16:26:48.630Z
SHA-256 of the archived text
78ee6ef371cf415f6bc2d36fb472f59aaf671d5fb830bbf6f2c22aff8d4f7e74
Stable URL of this page
https://metraipolis.com/documento/dc_4ecb9db3c094

How to cite

Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning — arXiv:2609.37066 — arXiv (en), https://arxiv.org/abs/2609.37066 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4ecb9db3c094 (Metraipolis archive dc_4ecb9db3c094, sha256 78ee6ef371cf415f6bc2d36fb472f59aaf671d5fb830bbf6f2c22aff8d4f7e74)
BibTeX
@misc{dc4ecb9db3c094,
  title = {Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning — arXiv:2609.37066},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37066},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4ecb9db3c094; sha256 78ee6ef371cf415f6bc2d36fb472f59aaf671d5fb830bbf6f2c22aff8d4f7e74},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37066. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.37066 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.936Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).