Metraipolis › archive › dc_573e72ca0a22

The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models — arXiv:2609.32679

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32679
License (as archived)
CC0-1.0
Archived
2026-10-07T21:00:32.449Z
SHA-256 of the archived text
248380df45efed42dc120f4382a83bf3b9d25c7cecafb3313ea1ae36caaee886
Stable URL of this page
https://metraipolis.com/documento/dc_573e72ca0a22

How to cite

The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models — arXiv:2609.32679 — arXiv (en), https://arxiv.org/abs/2609.32679 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_573e72ca0a22 (Metraipolis archive dc_573e72ca0a22, sha256 248380df45efed42dc120f4382a83bf3b9d25c7cecafb3313ea1ae36caaee886)
BibTeX
@misc{dc573e72ca0a22,
  title = {The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models — arXiv:2609.32679},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32679},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_573e72ca0a22; sha256 248380df45efed42dc120f4382a83bf3b9d25c7cecafb3313ea1ae36caaee886},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32679. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

GUI World Models (GUI-WMs) are increasingly used to predict future states for agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the vis- ible interface omits transition-relevant environment state, so identical observable conditions can correspond to different valid futures. To diagnose this failure mode, we introduce StateAliasBench, a diagnostic benchmark that explicitly isolates such ambiguities via strict pairing. We further propose lightweight predictive- state recovery that infers structured state from history and augments otherwise frozen GUI-WMs through a deterministic state interface. Family-specific special- ists provide state recovery across heterogeneous state types, and multi-teacher dis- tillation consolidates them into a single unified estimator. Experiments show that existing GUI-WMs exhibit systematic failures under observation-only condition- ing, while predictive-state augmentation substantially restores state-sensitive pre- diction across evaluated WMs, preserves generative fidelity, and improves down- stream performance of GUI agents on AndroidWorld. These results suggest that reliable GUI world modeling should account not only for what is visible, but also for the hidden transition state that determines what happens next.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.32679 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.435Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).