Metraipolis › archive › dc_cee87be169ad

SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation — arXiv:2608.14647

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.14647
License (as archived): CC0-1.0
Archived: 2026-10-18T17:51:16.012Z
SHA-256 of the archived text: 49f18c29d21d0f6542538c8fb4361ee3b0050521c68951992ad58b43447ed51f
Stable URL of this page: https://metraipolis.com/documento/dc_cee87be169ad

How to cite

SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation — arXiv:2608.14647 — arXiv (en), https://arxiv.org/abs/2608.14647 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cee87be169ad (Metraipolis archive dc_cee87be169ad, sha256 49f18c29d21d0f6542538c8fb4361ee3b0050521c68951992ad58b43447ed51f)

BibTeX

@misc{dccee87be169ad,
  title = {SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation — arXiv:2608.14647},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14647},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cee87be169ad; sha256 49f18c29d21d0f6542538c8fb4361ee3b0050521c68951992ad58b43447ed51f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14647. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dirty-history rollouts make multi-turn on-policy self-distillation (OPSD) brittle: once a student emits an erroneous intermediate reply, later turns are conditioned on that reply, and uniform distillation can spend loss on tokens that carry little corrective signal. We introduce SMOPD (Selective Masking for On-Policy Distillation), a loss-only stabilization method for multi-turn OPSD. For each generated middle-turn reply, SMOPD ranks token positions by student entropy and removes the lowest-entropy 20% from the clipped generalized Jensen-Shannon distillation loss; final-answer and FULL-preservation losses are unchanged. This design targets token-level uncertainty rather than coarse trajectory outcomes, adds no parameters, and has zero inference-time overhead. We compare SMOPD with a correctness-scaling variant that multiplies a common detached reliability proxy using final-answer correctness. On LiC with Qwen3 models, SMOPD improves SHARDED-view accuracy by 1.0-2.5 percentage points in single-seed 1.7B, 4B, and 8B comparisons, and a small 4B multi-seed check shows a +1.7pp mean SHARDED gain over baseline (two-tailed p = 0.022). Adding the outcome scalar is harmful without masking at 1.7B (-4.0pp) and remains scale-dependent when combined with masking (+1.3pp at 4B, neutral at 1.7B, and -0.5pp at 8B). These archived aggregate results suggest that token-level uncertainty is a more reliable stabilization signal than scalar final-answer correctness in this evaluated dirty-history OPSD setting, while leaving causal mechanism tests and broader benchmark validation to future work.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.14647 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.924Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).