Metraipolis › archive › dc_c2b4cebe0dae

MINT: Min-Selection Preference Distillation for Balanced Multi-Objective Alignment — arXiv:2608.14828

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14828
License (as archived)
CC0-1.0
Archived
2026-10-18T17:08:07.958Z
SHA-256 of the archived text
b4da6851cc96dea2c35c53768162c99835178740b0c6a26bb35e5326a8ea06fb
Stable URL of this page
https://metraipolis.com/documento/dc_c2b4cebe0dae

How to cite

MINT: Min-Selection Preference Distillation for Balanced Multi-Objective Alignment — arXiv:2608.14828 — arXiv (en), https://arxiv.org/abs/2608.14828 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2b4cebe0dae (Metraipolis archive dc_c2b4cebe0dae, sha256 b4da6851cc96dea2c35c53768162c99835178740b0c6a26bb35e5326a8ea06fb)
BibTeX
@misc{dcc2b4cebe0dae,
  title = {MINT: Min-Selection Preference Distillation for Balanced Multi-Objective Alignment — arXiv:2608.14828},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14828},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2b4cebe0dae; sha256 b4da6851cc96dea2c35c53768162c99835178740b0c6a26bb35e5326a8ea06fb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14828. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Aligning a language agent to several objectives at once is a persistent failure mode of preference-based training: when objectives are combined additively, optimization collapses onto whichever is cheapest to improve and sacrifices the rest, so a support agent learns to sound warm while giving no real help. The root issue is that an additive reward has no notion of balance. We introduce Mint (MIN-selection preference disTillation), a one-line change to preference distillation: rather than ranking sampled candidates by a weighted sum of rewards, we rank them by their weakest objective, distilling the best-balanced candidate over the most lopsided one with an unchanged DPO objective. This is the p -> negative infinity limit of a generalized-mean family spanning additive to worst-case selection. Across cooperative emotional support and adversarial negotiation, min-selection lifts both objectives while sharply cutting their imbalance; on emotional support it raises the weaker axis from 0.37 to 0.64 (p < 10^-40), surpassing human experts and persisting across full multi-turn rollouts. A turn-by-turn analysis yields our central finding: min-selection corrects imbalance in proportion to how imbalanced the reference policy is, and its benefit endures over an interaction precisely as long as that imbalance does.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.14828 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.740Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).