Metraipolis › archive › dc_018724db7207

DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance — arXiv:2608.14644

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14644
License (as archived)
CC0-1.0
Archived
2026-10-18T19:01:08.130Z
SHA-256 of the archived text
5cbb4430b8560d7970154f7c96ca20c9b1e3a9360733b19857bac12b4d37cde4
Stable URL of this page
https://metraipolis.com/documento/dc_018724db7207

How to cite

DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance — arXiv:2608.14644 — arXiv (en), https://arxiv.org/abs/2608.14644 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_018724db7207 (Metraipolis archive dc_018724db7207, sha256 5cbb4430b8560d7970154f7c96ca20c9b1e3a9360733b19857bac12b4d37cde4)
BibTeX
@misc{dc018724db7207,
  title = {DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance — arXiv:2608.14644},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14644},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_018724db7207; sha256 5cbb4430b8560d7970154f7c96ca20c9b1e3a9360733b19857bac12b4d37cde4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14644. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Real-world LLM deployments increasingly rely on runtime-injected prohibitions--enterprise policies, PII redlines, tool boundaries--that vary per request and per tenant. Conventional post-training is structurally ill-suited: SFT hides the violation signal in compliant labels, and DPO's sequence-level preferences mismatch token-localized violations. We propose DUET, a token-selective on-policy distillation method for prohibition compliance. DUET pairs a teacher that sees the prohibition (positive) with an identical-weight teacher that does not (negative). Because the two teachers differ only in prohibition visibility, their per-token disagreement isolates the prohibition's causal effect--yielding a clean supervision signal uncontaminated by model capacity or mismatch. This disagreement drives two complementary mechanisms: signal cleaning, which discards agreement tokens as redundant or prefix-corrupted, and preference-directed learning, which pushes the student away from the negative teacher and toward the positive one at token granularity, embedding DPO-style optimization directly into OPD without offline preference data. We construct an industrial Prohibition-Compliance benchmark spanning five task families covering explicit-refusal, paraphrase robustness, and over-refusal. Across 1.5B-8B Qwen variants, DUET achieves 72.3-85.2% violation compliance while preserving 88-93% normal utility, dramatically outperforming teacher model and other distillation baselines. External evaluation on SysBench confirms improved safety alignment with minimal degradation on GSM8K and MATH-500.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.14644 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T18:06:57.909Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).