Metraipolis › archive › dc_0fc20173d4e5

Marionette: Predicting World States, Rendering Geometry, Painting Appearance — arXiv:2608.14530

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.14530
License (as archived): CC0-1.0
Archived: 2026-10-19T02:21:38.234Z
SHA-256 of the archived text: e7fae8b4bc7b440e63d5be9c2fdc91b218a37201ce638f11523662008cbf5cb5
Stable URL of this page: https://metraipolis.com/documento/dc_0fc20173d4e5

How to cite

Marionette: Predicting World States, Rendering Geometry, Painting Appearance — arXiv:2608.14530 — arXiv (en), https://arxiv.org/abs/2608.14530 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0fc20173d4e5 (Metraipolis archive dc_0fc20173d4e5, sha256 e7fae8b4bc7b440e63d5be9c2fdc91b218a37201ce638f11523662008cbf5cb5)

BibTeX

@misc{dc0fc20173d4e5,
  title = {Marionette: Predicting World States, Rendering Geometry, Painting Appearance — arXiv:2608.14530},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14530},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0fc20173d4e5; sha256 e7fae8b4bc7b440e63d5be9c2fdc91b218a37201ce638f11523662008cbf5cb5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14530. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Interactive game world models typically autoregress visual observations directly in pixel or latent space, forcing structured properties such as pose, geometry, and occlusion to be implicitly maintained by the same generative sequence. Over long horizons, errors in these latent world properties accumulate, making consistency and controllability fragile. We explicitly model the evolving world state, delegate exact geometric computation to a fixed, zero-parameter renderer, and leave the neural model to synthesize appearance. We instantiate this idea as Marionette, a world model for interactive games with articulated characters. First, a two-stage autoregressive dynamics model predicts an explicit and interpretable 276-dimensional 3D world state comprising multi-entity articulated skeletons, metric root trajectories, and rotations. Second, a zero-parameter graphics bridge converts the predicted state into pose-control videos, computing world-space geometry and occlusion in closed form. Third, a control-conditioned video-diffusion observation model synthesizes photorealistic RGB observations from the resulting structured controls. Our experiments establish two properties of Marionette. First, the predicted world state is directly controllable. Forcing a mismatched action stream changes root-aligned joint error by 31% across 48 held-out segments. Second, long-horizon behaviour is determined in the state, and can be repaired there. Left free, the two generated characters drift to 21.2 m apart (recorded sessions stay near 5 m) and a third of frames show ground penetration. Two rules imposed on the explicit state, a terrain collider and a separation cap, cut penetration by 66% and keep the pair engaged, with no change to the observation model. Routing appearance through the predicted state costs no fidelity we can detect, at an FVD of 831 against 799 for recorded pose.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.14530 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T02:02:02.959Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).