Metraipolis › archive › dc_28432ea62cdf

ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning — arXiv:2609.35215

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35215
License (as archived)
CC0-1.0
Archived
2026-10-07T20:58:36.838Z
SHA-256 of the archived text
74764be9774e17719d21ba0472e6623ef79fecbace043eb7539e056fd2bf46ac
Stable URL of this page
https://metraipolis.com/documento/dc_28432ea62cdf

How to cite

ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning — arXiv:2609.35215 — arXiv (en), https://arxiv.org/abs/2609.35215 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_28432ea62cdf (Metraipolis archive dc_28432ea62cdf, sha256 74764be9774e17719d21ba0472e6623ef79fecbace043eb7539e056fd2bf46ac)
BibTeX
@misc{dc28432ea62cdf,
  title = {ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning — arXiv:2609.35215},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35215},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_28432ea62cdf; sha256 74764be9774e17719d21ba0472e6623ef79fecbace043eb7539e056fd2bf46ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35215. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Terminal utility evaluates a complete agentic workflow, but learning requires credit for the decisions within it. We introduce Attentive Search over Counterfactual Trees (ASCT), a framework that turns training-time multi-step search into local action credit. At actor-visited states, an auxiliary tree evaluates alternative legal actions from the same recoverable prefix. Its action-value table is centered by the frozen actor's probabilities and supplies credit for PPO on actor-sampled trajectories. This protocol connects counterfactual evaluation to policy learning while deploying the actor alone. Uniform, UCT, and cost-aware AgentUCT instantiate the framework. On HotpotQA agentic retrieval-augmented generation, all three improve mean held-out utility over trajectory-return PPO and workflow-adapted VinePPO. Across three seeds, ASCT-AgentUCT reaches 0.6187 utility versus 0.5939 for VinePPO, with gains in answer F1 and execution cost, and uses 50.3% fewer recorded auxiliary Qwen tokens. Transfer and component-description studies examine the learned policies beyond the training setting.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.35215 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.807Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).