Metraipolis › archive › dc_cfca6395af95

Beam Search as Test-Time Self-Distillation via Counterfactual Contexts — arXiv:2609.37041

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37041
License (as archived)
CC0-1.0
Archived
2026-10-07T16:26:46.376Z
SHA-256 of the archived text
30b06cf6650c8c00bcc97466b4a9cf63ab346786cfc03fdbfbfefe268316ab3a
Stable URL of this page
https://metraipolis.com/documento/dc_cfca6395af95

How to cite

Beam Search as Test-Time Self-Distillation via Counterfactual Contexts — arXiv:2609.37041 — arXiv (en), https://arxiv.org/abs/2609.37041 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cfca6395af95 (Metraipolis archive dc_cfca6395af95, sha256 30b06cf6650c8c00bcc97466b4a9cf63ab346786cfc03fdbfbfefe268316ab3a)
BibTeX
@misc{dccfca6395af95,
  title = {Beam Search as Test-Time Self-Distillation via Counterfactual Contexts — arXiv:2609.37041},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37041},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cfca6395af95; sha256 30b06cf6650c8c00bcc97466b4a9cf63ab346786cfc03fdbfbfefe268316ab3a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37041. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Self-Distillation Fine-Tuning (SDFT) enables a language model to act as its own teacher: by conditioning on a demonstration, the model produces an implicit reward via pointwise mutual information, which guides on-policy learning without external supervision. However, SDFT operates at training time: it requires gradient updates and access to expert demonstrations, making it inapplicable at inference. We propose test-time self-distillation, a decoding-time method that extracts a steering signal from the self-distillation framework without any parameter updates, reward models, or training data. Our key insight is that counterfactual contexts, i.e. fixed textual templates that hypothetically prime the model for excellent versus poor reasoning, can substitute for the demonstration. The log-odds ratio of a candidate answer under these two counterfactual conditions defines a new reward signal. We derive the optimal KL-regularized policy under this reward, which takes the form of a Gibbs reweighting of the base distribution. Crucially, this reweighting is global: it cannot be decomposed into independent per-token operations without ignoring future trajectory quality. We therefore approximate the target distribution via beam search. Experiments on mathematical reasoning (MATH500), code generation (HumanEval), and graduate-level science QA (GPQA) across multiple model scales show that test-time self-distillation improves over standard sampling, low temperature, beam search and power sampling baselines on average, demonstrating that the self-distillation principle can be operationalized at inference time.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.37041 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.933Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).