Metraipolis › archive › dc_643a093c5fc8
Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training — arXiv:2608.16926
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.16926
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:48:22.537Z
- SHA-256 of the archived text
a8f2b7206db9211733e1c6d0ce2b1c977e9a787c52e0989b15ab22ffa207fa7e- Stable URL of this page
https://metraipolis.com/documento/dc_643a093c5fc8
How to cite
Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training — arXiv:2608.16926 — arXiv (en), https://arxiv.org/abs/2608.16926 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_643a093c5fc8 (Metraipolis archive dc_643a093c5fc8, sha256 a8f2b7206db9211733e1c6d0ce2b1c977e9a787c52e0989b15ab22ffa207fa7e)
BibTeX
@misc{dc643a093c5fc8,
title = {Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training — arXiv:2608.16926},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.16926},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_643a093c5fc8; sha256 a8f2b7206db9211733e1c6d0ce2b1c977e9a787c52e0989b15ab22ffa207fa7e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16926. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Data selection in supervised fine-tuning aims to select a small set of effective samples from large-scale candidate data, reducing training cost while preserving model performance. However, existing methods usually treat data value as a relatively static property, and pay limited attention to the compatibility between data and the capability distribution of the target model. To address this issue, we propose Data-DPO, a target model-oriented SFT data selection method. Data-DPO observes the local training feedback of the target model on different samples through one-step probing, transforms activation differences among samples into pairwise data preferences, and trains a lightweight reward model to learn target-model-aware data preferences. In the final selection stage, Data-DPO further combines target model preference, external quality scores, and marginal diversity to construct a more stable and effective training subset. Experimental results on Vision-Flan and LLaVA-CoT show that Data-DPO consistently outperforms existing data selection baselines under multiple data budgets and stably surpasses full data training performance. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.16926 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:47:59.362Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).