Metraipolis › archive › dc_943382802b59
Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models — arXiv:2610.07704
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.07704
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T11:17:11.224Z
- SHA-256 of the archived text
089c1fa4fe35d7711e3f2eabee52cd9ea396e5528e3bdd0ef4c0079a8510f211- Stable URL of this page
https://metraipolis.com/documento/dc_943382802b59
How to cite
Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models — arXiv:2610.07704 — arXiv (en), https://arxiv.org/abs/2610.07704 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_943382802b59 (Metraipolis archive dc_943382802b59, sha256 089c1fa4fe35d7711e3f2eabee52cd9ea396e5528e3bdd0ef4c0079a8510f211)
BibTeX
@misc{dc943382802b59,
title = {Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models — arXiv:2610.07704},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.07704},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_943382802b59; sha256 089c1fa4fe35d7711e3f2eabee52cd9ea396e5528e3bdd0ef4c0079a8510f211},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07704. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Fully decentralized multi-agent reinforcement learning (MARL), also referred to as independent learning, requires each agent to learn and act using only its local information and experience, without a centralized critic or inter-agent communication. Such a stringent information structure renders the conventional reward signal ambiguous. A poor return may result from an ineffective ego action, an incompatible teammate response, or an effective opponent response, yet scalar rewards alone do not reveal which explanation is responsible. We argue that agents can learn more effectively by prospectively comparing the consequences of candidate actions rather than diagnosing failures only from realized returns. We introduce CASTLE (Counterfactual Action-conditioned Semantic Tokens for Local Execution in Decentralized MARL), an offline-training, online-in-context guidance framework with two complementary world models. A Local Dynamics World Model, offline pre-trained over agents' local trajectories, summarizes the agent's local trajectory dynamics and partial observability, while a Semantic-Social World Model predicts compact short-horizon task and social consequences for each candidate ego action. The latter is trained from counterfactual simulator rollouts that expose plausible teammate and opponent responses to alternative actions taken from the same logged rollout state. During online learning and execution, both world models remain frozen and are queried by agents using only locally available information. Their prediction logits provide in-context guidance to an independent PPO policy. Across 30 matched seeds on Tag, Spread, and Adversary in the benchmark multi-particle environments, our proposed CASTLE achieves the highest mean final score among the evaluated methods, exceeding the strongest baseline on each task by 10.67, 6.46, and 0.33 normalized points, respectively. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno). Categorie arXiv: Multiagent Systems (cs.MA) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.07704 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T11:15:09.458Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).