Metraipolis › archive › dc_095c16faad83

When Does Action Credit Need Updating? — arXiv:2609.29007

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29007
License (as archived)
CC0-1.0
Archived
2026-10-08T04:25:46.160Z
SHA-256 of the archived text
f446ab51b028dfd8d3201c7af0377792a6ef1172a48487fa723182b886a03e1a
Stable URL of this page
https://metraipolis.com/documento/dc_095c16faad83

How to cite

When Does Action Credit Need Updating? — arXiv:2609.29007 — arXiv (en), https://arxiv.org/abs/2609.29007 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_095c16faad83 (Metraipolis archive dc_095c16faad83, sha256 f446ab51b028dfd8d3201c7af0377792a6ef1172a48487fa723182b886a03e1a)
BibTeX
@misc{dc095c16faad83,
  title = {When Does Action Credit Need Updating? — arXiv:2609.29007},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29007},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_095c16faad83; sha256 f446ab51b028dfd8d3201c7af0377792a6ef1172a48487fa723182b886a03e1a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29007. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Tool-using agents are continually updated with new interaction data. After each policy update, however, previously estimated action credits may become stale. Recomputing them from scratch can require many additional tool calls and environment interactions, making repeated updates increasingly expensive. We ask a simple question: when does historical action credit actually need to be updated? Our key observation is that a change in action value does not necessarily imply a change in the decision. Historical credit can still be useful as long as policy-induced drift is too small to overturn the existing action ranking. Building on this idea, we introduce pairwise branch sensitivity to capture how strongly a policy update affects the downstream regions that distinguish two candidate actions. We then derive a first-order anchored credit-transport estimator that updates historical credit using old interventional trajectories, and propose a Decision-Sufficient Credit Gate (DSC-Gate) that chooses whether to reuse, transport, or resample credit. Experiments show that branch sensitivity explains credit drift substantially better than global policy distance. With sufficient historical data, credit transport reduces estimation error, while its benefit to decision making is concentrated on updates that affect action-distinguishing branches. On a fully independent test set, DSC-Gate changes mean regret by only +0.00004 relative to a gap-based gate while reducing mean new tool steps from 472 to 286, a 39.4% reduction. We observe the same pattern after a real tool-agent parameter update. Overall, our results show that agents do not need to recompute action credit after every policy update: much of the historical evidence can be reused or cheaply corrected, reducing the additional interaction required to keep action decisions up to date.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.29007 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.697Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).