Metraipolis › archive › dc_c83dfb3a67ae

Iterative Policy Refinement through Semantic Rollout Analysis — arXiv:2610.01652

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.01652
License (as archived): CC0-1.0
Archived: 2026-10-06T12:14:44.323Z
SHA-256 of the archived text: 6fd0c719d0125024cf1a2cdda21ce0dfa6db9f789e4bc151b4964dd3ebd9a2d2
Stable URL of this page: https://metraipolis.com/documento/dc_c83dfb3a67ae

How to cite

Iterative Policy Refinement through Semantic Rollout Analysis — arXiv:2610.01652 — arXiv (en), https://arxiv.org/abs/2610.01652 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c83dfb3a67ae (Metraipolis archive dc_c83dfb3a67ae, sha256 6fd0c719d0125024cf1a2cdda21ce0dfa6db9f789e4bc151b4964dd3ebd9a2d2)

BibTeX

@misc{dcc83dfb3a67ae,
  title = {Iterative Policy Refinement through Semantic Rollout Analysis — arXiv:2610.01652},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01652},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c83dfb3a67ae; sha256 6fd0c719d0125024cf1a2cdda21ce0dfa6db9f789e4bc151b4964dd3ebd9a2d2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01652. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Structured policies improve efficiency, robustness, and interpretability in imitation learning by introducing task-specific inductive bias, but existing structure generation methods rely either on extensive human input or on static domain knowledge encoded in LLMs, which may be inconsistent with the expert demonstrations. We propose a closed-loop framework that iteratively refines structured policies using LLM-guided analysis of policy rollouts. By logging rollouts as semantically meaningful tabular data and prompting the LLM to generate diagnostic analysis code, our method identifies suboptimalities in the policy structure and iteratively corrects them without requiring human instruction. Experiments on car racing and door opening tasks show that our approach improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance. These results demonstrate that tabular rollout analysis provides an effective feedback signal to align LLM-generated policy structures with expert demonstrations, and we can utilize it to generate good policy structures automatically.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.01652 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.207Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).