Metraipolis › archive › dc_d434aff907eb

Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection — arXiv:2608.20169

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.20169
License (as archived)
CC0-1.0
Archived
2026-10-18T11:39:23.793Z
SHA-256 of the archived text
a36ce238d79146236ea61bd27ac2d3ca84a7842d8d0d21f898478be57864ef9d
Stable URL of this page
https://metraipolis.com/documento/dc_d434aff907eb

How to cite

Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection — arXiv:2608.20169 — arXiv (en), https://arxiv.org/abs/2608.20169 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d434aff907eb (Metraipolis archive dc_d434aff907eb, sha256 a36ce238d79146236ea61bd27ac2d3ca84a7842d8d0d21f898478be57864ef9d)
BibTeX
@misc{dcd434aff907eb,
  title = {Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection — arXiv:2608.20169},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20169},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d434aff907eb; sha256 a36ce238d79146236ea61bd27ac2d3ca84a7842d8d0d21f898478be57864ef9d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20169. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose $\textbf{Task-CoEvolve}$, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.20169 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T11:38:27.932Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).