Metraipolis › archive › dc_5f8ae2e55671
Discovering Learning Rules that Shape Others — arXiv:2611.08335
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08335
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:50:57.540Z
- SHA-256 of the archived text
519e3e7daa4c7c1fb867dbde61902f686e56ac8368260fbc4b56a6d5f6763f1c- Stable URL of this page
https://metraipolis.com/documento/dc_5f8ae2e55671
How to cite
Discovering Learning Rules that Shape Others — arXiv:2611.08335 — arXiv (en), https://arxiv.org/abs/2611.08335 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5f8ae2e55671 (Metraipolis archive dc_5f8ae2e55671, sha256 519e3e7daa4c7c1fb867dbde61902f686e56ac8368260fbc4b56a6d5f6763f1c)
BibTeX
@misc{dc5f8ae2e55671,
title = {Discovering Learning Rules that Shape Others — arXiv:2611.08335},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08335},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5f8ae2e55671; sha256 519e3e7daa4c7c1fb867dbde61902f686e56ac8368260fbc4b56a6d5f6763f1c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08335. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In general-sum games, self-interested learners can reach outcomes that are poor for everyone, which has motivated methods that shape the opponent's learning. Methods like Learning with Opponent-Learning Awareness (LOLA), which differentiates through the opponent's learning process, and Model-Free Opponent Shaping (M-FOS), which uses model-free optimisation methods for long-horizon opponent shaping, have enabled mutual cooperation and exploitation of learners. However, these methods require access to the opponent's parameters, policy, rewards or learning rule during deployment. In single-agent reinforcement learning, update rules discovered by meta-learning have achieved state-of-the-art results (e.g. DiscoRL). These rules are separate from the policy they train and read only the agent's own experience. We hereby extend this framework to general-sum settings by introducing Discovered Update Rule for Opponent Shaping (DUOS). To the best of our knowledge, this is the first method to discover an update rule for an agent in general-sum games. DUOS meta-trains the rule to maximise the agent's expected return over its lifetime, and updates an inner network using the agent's own experience. We show that our discovered rule exhibits behaviour consistent with opponent shaping without reading opponent policies or parameters, and ranks 1st in normalised mean score and 1st in self-play among tournament entrants. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.08335 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:21:42.647Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).