Metraipolis › archive › dc_94f984e34524

PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State — arXiv:2611.07867

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.07867
License (as archived)
CC0-1.0
Archived
2026-11-06T14:51:15.543Z
SHA-256 of the archived text
ca80e1a18b467eb5335d04dd05f8389b55ddddcb4583cc4ab78c03d3ded6a047
Stable URL of this page
https://metraipolis.com/documento/dc_94f984e34524

How to cite

PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State — arXiv:2611.07867 — arXiv (en), https://arxiv.org/abs/2611.07867 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_94f984e34524 (Metraipolis archive dc_94f984e34524, sha256 ca80e1a18b467eb5335d04dd05f8389b55ddddcb4583cc4ab78c03d3ded6a047)
BibTeX
@misc{dc94f984e34524,
  title = {PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State — arXiv:2611.07867},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07867},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_94f984e34524; sha256 ca80e1a18b467eb5335d04dd05f8389b55ddddcb4583cc4ab78c03d3ded6a047},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07867. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Building a playable world requires more than generating plausible video: it demands responsive control, persistent spatial structure, and coherent interactions with dynamic entities. Using generated frames as the sole recurrent world representation ties these requirements to view-dependent appearance synthesis, making it difficult to maintain long-term spatial and temporal consistency in generation and interaction. We present PlayGen2, a real-time interactive world model that separates learned 3D state evolution from neural rendering. PlayGen2 maintains a persistent world memory comprising static geometry represented by a volumetric occupancy grid and identity-indexed 3D states for the player and other entities. Within a fixed-size local region, its world-state model jointly learns action-conditioned entity dynamics, geometry completion, and entity discovery, committing updates to a persistent world memory. A separate neural renderer synthesizes first-person observations from depth and semantic projections of the world state, without feeding generated images back into state prediction. Using Doom as a testbed, we demonstrate that the model successfully learns world dynamics including geometry-aware player motion and player--NPC interactions. The framework is computationally efficient, enabling the complete simulation and rendering pipeline to achieve approximately 160ms backend latency on a MacBook Pro with an M4 Pro chip. Beyond single-player interaction, we demonstrate zero-shot deployment on unseen Doom maps, multiplayer interaction through the composition of single-player models, and programmable gameplay through rule-based state interventions. These capabilities demonstrate the transferability and extensibility of the learned game-world model. Code and pretrained model weights are publicly available.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.07867 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:21:42.664Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).