Metraipolis › archive › dc_fface4cdc355

DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies — arXiv:2610.06993

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.06993
License (as archived): CC0-1.0
Archived: 2026-10-07T11:20:06.912Z
SHA-256 of the archived text: 391b06117ba9906746aed40b8c3628f27bb3ce8bee9be4bdabc25cb50d565497
Stable URL of this page: https://metraipolis.com/documento/dc_fface4cdc355

How to cite

DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies — arXiv:2610.06993 — arXiv (en), https://arxiv.org/abs/2610.06993 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fface4cdc355 (Metraipolis archive dc_fface4cdc355, sha256 391b06117ba9906746aed40b8c3628f27bb3ce8bee9be4bdabc25cb50d565497)

BibTeX

@misc{dcfface4cdc355,
  title = {DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies — arXiv:2610.06993},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.06993},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fface4cdc355; sha256 391b06117ba9906746aed40b8c3628f27bb3ce8bee9be4bdabc25cb50d565497},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.06993. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Evolution Strategies (ES) enable memory efficient full parameter fine-tuning of large language models (LLMs) using only forward computation. However, standard ES uniformly averages rewards across problems and compresses problem level population feedback into a single scalar, making it difficult to capture how the learning value of each problem changes with model capability. To address this limitation, we propose Difficulty-Aware Reweighting and Targeted Replay for Evolution Strategies (DART-ES). DART-ES estimates the local solvability of each problem from its pass rate across the perturbation population and aggregates historical observations to construct a dynamic difficulty state. This shared state jointly guides continuous difficulty reweighting and rare solvable sample replay, thereby improving perturbation direction evaluation and training data allocation without introducing an additional difficulty model or backpropagation. Extensive experiments show that DART-ES achieves good fine-tuning performance. DART-ES outperforms ES on all five base models and improves the average accuracy from 72.07\% to 73.53\%, exceeding the 73.26\% achieved by GRPO on GSM8K. Across five challenging mathematical reasoning benchmarks, DART-ES achieves an average accuracy of 49.20\%, compared with 48.34\% for ES and remains competitive with strong 7B models trained with RL. Further experiments show consistent gains in instruction tuning, code generation and the Countdown task with a 14B model, demonstrating strong generalization across tasks and scalability to larger models. Beyond performance gains, DART-ES also shows clear advantages in system efficiency. It reduces runtime per step by 15.2\%--50.2\% and peak memory usage per GPU by 21.1\%--51.1\% compared with GRPO. Despite performing full parameter fine-tuning, DART-ES also requires less runtime and GPU memory than GRPO+LoRA.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.06993 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.654Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).