Metraipolis › archive › dc_c70aa418ffc8

Decision-Focused Learning in MDPs: An Occupancy Measure Approach — arXiv:2610.08384

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.08384
License (as archived): CC0-1.0
Archived: 2026-10-07T11:41:46.021Z
SHA-256 of the archived text: d89f2375186795f6203449aa2018dbce15548b8c8e26cffb58f93c4ce67fff13
Stable URL of this page: https://metraipolis.com/documento/dc_c70aa418ffc8

How to cite

Decision-Focused Learning in MDPs: An Occupancy Measure Approach — arXiv:2610.08384 — arXiv (en), https://arxiv.org/abs/2610.08384 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c70aa418ffc8 (Metraipolis archive dc_c70aa418ffc8, sha256 d89f2375186795f6203449aa2018dbce15548b8c8e26cffb58f93c4ce67fff13)

BibTeX

@misc{dcc70aa418ffc8,
  title = {Decision-Focused Learning in MDPs: An Occupancy Measure Approach — arXiv:2610.08384},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08384},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c70aa418ffc8; sha256 d89f2375186795f6203449aa2018dbce15548b8c8e26cffb58f93c4ce67fff13},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08384. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In this work, we consider decision-focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT conditions of the Bellman equation and require solving a linear system over all state-action pairs, limiting its scalability. We address this by reformulating the MDP as an occupancy measure-based linear program (LP), whose feasible region is induced by predicted dynamics, and we derive a closed-form gradient by identifying the active constraints in the feasible polyhedron via the pivoting algorithm. This occupancy measure-based LP layer raises two challenges: (1) LP's solution gradient is discontinuous when active constraints change, and (2) the LP backward cost still scales with the state size, which is costly for large or continuous state spaces. We address the challenges with an augmented Lagrangian surrogate and smooth the boundary jumps by random row sketching of the constraints, and a learnable soft state-aggregation layer and its function-approximation generalization that scales the LP to large finite and continuous-state MDPs. Across multiple tasks, our methods reach lower regret than KKT-based DFL and two-stage baselines with significantly lower computation cost. The source code for all experiments is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.08384 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.828Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).