Metraipolis › archive › dc_88dbb4bd8ca8

Generative Policy Improvement from Observational Data — arXiv:2610.23436

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.23436
License (as archived)
CC0-1.0
Archived
2026-10-19T08:42:36.974Z
SHA-256 of the archived text
42e26132626f122ef4ffabf4affa719306bf35cad900e3d0f044d2101b063f63
Stable URL of this page
https://metraipolis.com/documento/dc_88dbb4bd8ca8

How to cite

Generative Policy Improvement from Observational Data — arXiv:2610.23436 — arXiv (en), https://arxiv.org/abs/2610.23436 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_88dbb4bd8ca8 (Metraipolis archive dc_88dbb4bd8ca8, sha256 42e26132626f122ef4ffabf4affa719306bf35cad900e3d0f044d2101b063f63)
BibTeX
@misc{dc88dbb4bd8ca8,
  title = {Generative Policy Improvement from Observational Data — arXiv:2610.23436},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.23436},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_88dbb4bd8ca8; sha256 42e26132626f122ef4ffabf4affa719306bf35cad900e3d0f044d2101b063f63},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23436. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study how to improve decision policies from observational data to generate better actions when actions are complex, structured objects, such as text messages or images. This task is subject to two major challenges: (i) action-level overlap is often too weak for reliable off-policy evaluation and improvement, and (ii) generating new structured actions often requires expensive training or fine-tuning (e.g., of an LLM) and can be ill-defined due to information loss in action representations. To address both, we propose PolicyLift, a three-stage framework for generative policy improvement with structured actions from observational data. In Stage 1, PolicyLift learns a small set of discrete action codes that capture high-level strategies in the observed action space and are tailored to support policy improvement under limited overlap. In Stage 2, it learns a context-dependent policy over these action codes. At deployment time, Stage 3 "lifts" the learned code-level policy back to the original action space by generating actions from the historical behavior system in a code-conditioned way. We show theoretically that improving the policy over action codes guarantees improvement of the induced policy in the original action space. We demonstrate PolicyLift across experiments in synthetic and real-world text generation tasks with consistent policy improvements over the behavior policy.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.23436 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:10.081Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).