Metraipolis › archive › dc_e1fcd7cc4e5a

When Should Forecasting Agents Reason? Behavioral Stress Tests for Reliability Routing — arXiv:2609.28475

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.28475
License (as archived): CC0-1.0
Archived: 2026-10-08T04:25:36.274Z
SHA-256 of the archived text: be39435cd2daa1b20ccc4010cccf872c94be034c86ed464bb2a37f039b776b44
Stable URL of this page: https://metraipolis.com/documento/dc_e1fcd7cc4e5a

How to cite

When Should Forecasting Agents Reason? Behavioral Stress Tests for Reliability Routing — arXiv:2609.28475 — arXiv (en), https://arxiv.org/abs/2609.28475 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e1fcd7cc4e5a (Metraipolis archive dc_e1fcd7cc4e5a, sha256 be39435cd2daa1b20ccc4010cccf872c94be034c86ed464bb2a37f039b776b44)

BibTeX

@misc{dce1fcd7cc4e5a,
  title = {When Should Forecasting Agents Reason? Behavioral Stress Tests for Reliability Routing — arXiv:2609.28475},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.28475},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e1fcd7cc4e5a; sha256 be39435cd2daa1b20ccc4010cccf872c94be034c86ed464bb2a37f039b776b44},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28475. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Forecasting agents increasingly combine language-model reasoning, retrieval, ensembling, and calibration, but it remains unclear when each behavior should be trusted. We study this question on ForecastBench-style binary forecasting tasks, treating the choice to retrieve, reason, defer to a market prior, or use a historical analog as an observable agent behavior rather than a hidden implementation detail. Our central finding is that mechanism choice is source-dependent: structured analogs dominate for some data-generating processes, while market/crowd-style and conservative baselines are better for others. We introduce ReliabilityRoute, a structural intervention that steers forecasting-agent behavior using reliability features such as historical coverage, market-prior availability, source-prior sharpness, evidence strength, evidence disagreement, and horizon. A fixed 2024-fitted rule closely matches a hand taxonomy without hard-coded source-name decisions, while a walk-forward self-adjusting rule refits thresholds from previously resolved vintages and obtains the best mean Brier score among our deterministic systems across 16 later LLM vintages. The gain is modest and historical/search baselines remain highly competitive. The main contribution is therefore a behavioral stress test showing that more reasoning is not always better; forecasting agents should first estimate which evidence source deserves control, routing policies should themselves adapt under auditable constraints, and reproducibility artifacts are available at this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.28475 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.643Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).