Metraipolis › archive › dc_e7d4486fbb82

From Single-Agent Self-Discovery to Multi-Agent Adaptation: Scaling Laws of Partner Feedback and Convention Rigidity in Meta Reinforcement Learning Algorithms — arXiv:2611.01742

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01742
License (as archived)
CC0-1.0
Archived
2026-11-03T18:09:37.762Z
SHA-256 of the archived text
e3aa3216431ebb413042e455f31843f4f6e4880d8c19db8ed938d977b4ca5623
Stable URL of this page
https://metraipolis.com/documento/dc_e7d4486fbb82

How to cite

From Single-Agent Self-Discovery to Multi-Agent Adaptation: Scaling Laws of Partner Feedback and Convention Rigidity in Meta Reinforcement Learning Algorithms — arXiv:2611.01742 — arXiv (en), https://arxiv.org/abs/2611.01742 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e7d4486fbb82 (Metraipolis archive dc_e7d4486fbb82, sha256 e3aa3216431ebb413042e455f31843f4f6e4880d8c19db8ed938d977b4ca5623)
BibTeX
@misc{dce7d4486fbb82,
  title = {From Single-Agent Self-Discovery to Multi-Agent Adaptation: Scaling Laws of Partner Feedback and Convention Rigidity in Meta Reinforcement Learning Algorithms — arXiv:2611.01742},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01742},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e7d4486fbb82; sha256 e3aa3216431ebb413042e455f31843f4f6e4880d8c19db8ed938d977b4ca5623},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01742. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A key milestone toward Recursive Self-Improvement (RSI) is replacing hand-crafted optimizers with meta-learned update rules (such as Disco103 and OPEN) that discover how agents learn. Yet when a self-discovered RL model trains with adapting teammates, does co-evolution sustain capability gains or trap agents in brittle conventions? Across 1,800+ runs in continuous collective robotics (VMAS) and discrete coordination (Overcooked), we establish five laws of multi-agent adaptation. First, four-cell counterfactual partner swaps expose an evaluation illusion: apparent team progress often masks focal regression (+1.066 team gain concealing -0.119 focal loss) and inflates transfer by +0.649 to +1.210. Second, a 288-run physical-coupling sweep ($\alpha \in [0, 1.5]$) reveals a linear scaling law ($\beta = +0.255$, $p = 0.000488$, Spearman $\rho = 1.0$): live reciprocal feedback keeps Disco103 invariant (AUC 0.297--0.322), whereas severing feedback triggers a monotonic collapse (-0.172 to -0.490), with OPEN establishing a sensitivity spectrum (Disco103 > OPEN > MAPPO). Third, extending navigation training eightfold (1.05M steps) enables Disco103 to overtake MAPPO (+0.087 team return, +0.133 bank gain), while decoupled training boosts cross-partner completion by +10.16% (56.25% vs. 46.09%). Fourth, in Overcooked, MAPPO's self-play conventions induce a 28.13% cross-partner collision rate, whereas Disco103 cuts collisions to 12.77% and achieves 3.33x higher bank return (9.96 vs. 2.99; net gain +6.43), further improved by decoupled meta-adaptation (+1.39 with scripted partners). Finally, perturbation pulses ($p=0.00098$), auxiliary entropy diagnostics (8.485 nats), and 48 multi-lifetime meta-evolved rules ($L_2 \in [2.11, 4.91]$) trace these dynamics to partner-gradient blindness and auxiliary drift, informing multi-agent self-discoverd algorithms.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2611.01742 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:37.668Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).