Metraipolis › archive › dc_f362b66bf9e2

Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution — arXiv:2610.35733

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.35733
License (as archived)
CC0-1.0
Archived
2026-10-27T07:55:30.275Z
SHA-256 of the archived text
92cef2af7cebca812ba61c8555eb403e7c79fcb2dcb62afb7ddd3ebffb55ae8e
Stable URL of this page
https://metraipolis.com/documento/dc_f362b66bf9e2

How to cite

Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution — arXiv:2610.35733 — arXiv (en), https://arxiv.org/abs/2610.35733 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f362b66bf9e2 (Metraipolis archive dc_f362b66bf9e2, sha256 92cef2af7cebca812ba61c8555eb403e7c79fcb2dcb62afb7ddd3ebffb55ae8e)
BibTeX
@misc{dcf362b66bf9e2,
  title = {Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution — arXiv:2610.35733},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.35733},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f362b66bf9e2; sha256 92cef2af7cebca812ba61c8555eb403e7c79fcb2dcb62afb7ddd3ebffb55ae8e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35733. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-objective reinforcement learning is a fundamental problem for modern AI, with applications spanning post-training, reasoning models, and robotic control. A central challenge is to learn policies that balance competing objectives, such as helpfulness versus honesty in safety and alignment, or speed versus energy in robotics. The optimal policy can vary sharply with objective-weighting combinations, and the extent of this variation depends crucially on the degree of conflict among objectives, making per-preference policy learning sample-inefficient. In this paper, we introduce objective-conflict coefficients and derive near-optimal regret bounds for online settings using this measure, thereby quantifying the impact of misalignment on learning efficiency. Then, we design routed ensemble policy learning, an algorithm that maintains a small set of basis policies and dynamically routes trajectories to the policy with the highest potential reward. Finally, we consider trajectory selection as another approach to mitigate objective conflict, and design a kernel surrogate trajectory attribution algorithm that estimates how feedback signals from a set of trajectories influence policy learning on other trajectories. The kernel can capture nonlinear interactions such as synergy and antagonism between trajectories. We evaluate our approach across diverse environments and benchmarks: routed ensembles improve average performance by 4.3% over baselines for online environments from Meta-World and MO-Gymnasium, and kernel surrogate attribution improves correlation with leave-one-out analysis over existing baselines by 50%. When applied to trajectory selection, kernel-based attribution yields a 27% improvement in reward over baseline attribution methods.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Robotics (cs.RO); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.35733 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:15.067Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).