Metraipolis › archive › dc_2059e146bb45

Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts — arXiv:2609.33257

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33257
License (as archived)
CC0-1.0
Archived
2026-10-07T22:37:19.378Z
SHA-256 of the archived text
90c29c3bf297bb9af7620616032c2bdd4b4580a5a84b3330ede348ad493444a3
Stable URL of this page
https://metraipolis.com/documento/dc_2059e146bb45

How to cite

Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts — arXiv:2609.33257 — arXiv (en), https://arxiv.org/abs/2609.33257 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2059e146bb45 (Metraipolis archive dc_2059e146bb45, sha256 90c29c3bf297bb9af7620616032c2bdd4b4580a5a84b3330ede348ad493444a3)
BibTeX
@misc{dc2059e146bb45,
  title = {Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts — arXiv:2609.33257},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33257},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2059e146bb45; sha256 90c29c3bf297bb9af7620616032c2bdd4b4580a5a84b3330ede348ad493444a3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33257. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used to forecast real-world events, but access to the strongest individual forecaster may be costly or otherwise constrained. We study weak-to-strong forecast aggregation: can individually weaker LLM forecasters be aggregated to outperform a stronger forecaster? Using ForecastBench (Karger et al., 2025), we evaluate 70 LLM forecasters across 16 comparison groups, each with more than 1,000 shared subquestions, yielding 1,121 weaker-model pairs. Within each group, we identify the strongest individual by test Brier score and evaluate aggregates composed exclusively of weaker forecasters, with aggregation weights learned on separate training data. We find substantial evidence of weak-to-strong improvement. Learned linear pooling identifies a weaker pair that matches or outperforms the strongest individual in 11 of 16 groups and comes within 5% of its Brier score in all 16 groups. We also find that these improvements do not rely on having a near-best constituent and are generally accompanied by good calibration. Additional analyses show that adding more models does not consistently improve performance, and competitive weaker-model aggregates also remain available under practical constraints.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.33257 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.909Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).