Metraipolis › archive › dc_ec182a555b09
SAPD: Learning from Agent Experience without Environment Interaction — arXiv:2610.27847
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.27847
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:06:53.138Z
- SHA-256 of the archived text
2fec50fd5465ad48ffe769b50839f71edfa5568f0389227d1fe1aa927e6b1f44- Stable URL of this page
https://metraipolis.com/documento/dc_ec182a555b09
How to cite
SAPD: Learning from Agent Experience without Environment Interaction — arXiv:2610.27847 — arXiv (en), https://arxiv.org/abs/2610.27847 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec182a555b09 (Metraipolis archive dc_ec182a555b09, sha256 2fec50fd5465ad48ffe769b50839f71edfa5568f0389227d1fe1aa927e6b1f44)
BibTeX
@misc{dcec182a555b09,
title = {SAPD: Learning from Agent Experience without Environment Interaction — arXiv:2610.27847},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.27847},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec182a555b09; sha256 2fec50fd5465ad48ffe769b50839f71edfa5568f0389227d1fe1aa927e6b1f44},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27847. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Scaling the training of Large Language Model (LLM) agents typically requires a large collection of executable environments, whose construction, maintenance, and operation constrain the expansion of training experience. Existing offline trajectories offer an opportunity to reuse interaction experience, yet effectively refining the current policy without revisiting these environments remains challenging: supervised fine-tuning (SFT) can lead to catastrophic forgetting, while online reinforcement learning requires real-time interaction with the environment. To address this challenge, we propose Step-Advantage Policy Distillation (SAPD), a method for learning from offline trajectories using relative step advantages. SAPD decomposes multi-turn trajectories into individual decision contexts for on-policy distillation, allowing the student to generate responses conditioned on recorded histories, and introduces relative step advantage signals to stabilize training. The entire process requires neither new environment observations nor additional multi-turn interactions. Experiments on ALFWorld, WebShop, and Search-based QA show that SAPD improves performance across different teacher configurations, mitigates catastrophic forgetting compared with SFT, and runs at least $4.7\times$ faster per training step than online OPD. We will release our code soon. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.27847 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:43.449Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).