Metraipolis › archive › dc_46a2b220bdb8
MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.00865
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:41:46.393Z
- SHA-256 of the archived text
e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad- Stable URL of this page
https://metraipolis.com/documento/dc_46a2b220bdb8
How to cite
MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865 — arXiv (en), https://arxiv.org/abs/2609.00865 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46a2b220bdb8 (Metraipolis archive dc_46a2b220bdb8, sha256 e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad)
BibTeX
@misc{dc46a2b220bdb8,
title = {MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.00865},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46a2b220bdb8; sha256 e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00865. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Production agent harnesses such as Claude Code and Qwen-Agent compress context during rollout, but training under compression creates a conditioning problem: every eviction branches the effective history, so the learning object is a tree rather than a sequence. Existing linearizations either retain the rightmost path, causing time-travel leakage, or replay a depth-first traversal, causing train-inference mismatch. We introduce two exact, gradient-equivalent corrections: LogitTree, a segmented K-forward traversal, and a packed 4D attention mask. LogitTree requires K+1 backward passes; the 4D mask requires a custom kernel and white-box eviction records. We also propose SDCC (Self-Distillation for Conditioning Consistency), a single-backward-pass variational relaxation. At each eviction, it minimizes forward KL between the compressed student and a stop-gradient teacher on the reconstructed pre-eviction prefix. A residual per-junction KL of epsilon_KL gives an O(sqrt(epsilon_KL)) bound on the train-deployment total-variation gap. SDCC also applies to black-box harnesses. On seven web-search benchmarks with TC-RAG, AgentFold, MemexRL, Claude Code, and OpenCode, naive training inflates the train-rollout log-probability gap, especially on eviction-heavy batches. The exact methods stay at the no-compression floor, and SDCC substantially closes the gap, with lower logit drift and higher rollout rewards. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.00865 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T06:33:38.891Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).