Metraipolis › archive › dc_46a2b220bdb8

MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.00865
License (as archived)
CC0-1.0
Archived
2026-10-17T06:41:46.393Z
SHA-256 of the archived text
e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad
Stable URL of this page
https://metraipolis.com/documento/dc_46a2b220bdb8

How to cite

MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865 — arXiv (en), https://arxiv.org/abs/2609.00865 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46a2b220bdb8 (Metraipolis archive dc_46a2b220bdb8, sha256 e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad)
BibTeX
@misc{dc46a2b220bdb8,
  title = {MemoryWalker: Stop Training Agents on Contexts They Never Saw — arXiv:2609.00865},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00865},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46a2b220bdb8; sha256 e4d09acec94944f523c34c6fccc46f2b5980767b04ec614957d08b5d80cff0ad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00865. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Production agent harnesses such as Claude Code and Qwen-Agent compress context during rollout, but training under compression creates a conditioning problem: every eviction branches the effective history, so the learning object is a tree rather than a sequence. Existing linearizations either retain the rightmost path, causing time-travel leakage, or replay a depth-first traversal, causing train-inference mismatch. We introduce two exact, gradient-equivalent corrections: LogitTree, a segmented K-forward traversal, and a packed 4D attention mask. LogitTree requires K+1 backward passes; the 4D mask requires a custom kernel and white-box eviction records. We also propose SDCC (Self-Distillation for Conditioning Consistency), a single-backward-pass variational relaxation. At each eviction, it minimizes forward KL between the compressed student and a stop-gradient teacher on the reconstructed pre-eviction prefix. A residual per-junction KL of epsilon_KL gives an O(sqrt(epsilon_KL)) bound on the train-deployment total-variation gap. SDCC also applies to black-box harnesses. On seven web-search benchmarks with TC-RAG, AgentFold, MemexRL, Claude Code, and OpenCode, naive training inflates the train-rollout log-probability gap, especially on eviction-heavy batches. The exact methods stay at the no-compression floor, and SDCC substantially closes the gap, with lower logit drift and higher rollout rewards.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.00865 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:33:38.891Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).