Metraipolis › archive › dc_d0d16618da3e

Marginally Correct Tool Caches Can Reverse Group-Normalized Policy Updates — arXiv:2609.26866

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.26866
License (as archived): CC0-1.0
Archived: 2026-10-08T10:51:55.409Z
SHA-256 of the archived text: cb9225fb08d16dbf00ba7d0c3c7ff2c1da910c52262312447879fe1d7866fb13
Stable URL of this page: https://metraipolis.com/documento/dc_d0d16618da3e

How to cite

Marginally Correct Tool Caches Can Reverse Group-Normalized Policy Updates — arXiv:2609.26866 — arXiv (en), https://arxiv.org/abs/2609.26866 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d0d16618da3e (Metraipolis archive dc_d0d16618da3e, sha256 cb9225fb08d16dbf00ba7d0c3c7ff2c1da910c52262312447879fe1d7866fb13)

BibTeX

@misc{dcd0d16618da3e,
  title = {Marginally Correct Tool Caches Can Reverse Group-Normalized Policy Updates — arXiv:2609.26866},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26866},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d0d16618da3e; sha256 cb9225fb08d16dbf00ba7d0c3c7ff2c1da910c52262312447879fe1d7866fb13},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26866. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Tool-result caching reduces repeated execution in agent training, but also couples rollout randomness. We study a two-action model in which independent and shared execution preserve every rollout's conditional reward distribution. Despite this marginal agreement, sharing one stochastic result per group can reverse the expected group-normalized policy update. We derive an exact finite-group expression: against a constant alternative, the shared update follows the probability of winning minus the probability of losing, rather than the difference in expected reward. A Bernoulli specialization yields a wrong-direction region and a non-vanishing update-variance floor as group size grows. Centering without group standard-deviation scaling preserves the expected-return direction in this model, using an existing estimator control. Exhaustive finite sums verify 540 configurations and 3,240 estimator evaluations, with a separate ordered-sequence checker. An implementation audit reproduces the sharing path in a pinned, unmodified TVCache stack using 256 scripted rollouts. These results do not measure language-model training performance or refute TVCache's deterministic-output contract. They establish that marginal output validity alone cannot certify a stochastic cache as training-equivalent.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.26866 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:06:16.681Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).