Metraipolis › archive › dc_8d7371c8e91c

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations — arXiv:2610.22708

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.22708
License (as archived): CC0-1.0
Archived: 2026-10-19T08:42:12.502Z
SHA-256 of the archived text: b4dce5de19c1c1733e80b0eec8e525b06ebd7eab708e14ece0182a729da6b814
Stable URL of this page: https://metraipolis.com/documento/dc_8d7371c8e91c

How to cite

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations — arXiv:2610.22708 — arXiv (en), https://arxiv.org/abs/2610.22708 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8d7371c8e91c (Metraipolis archive dc_8d7371c8e91c, sha256 b4dce5de19c1c1733e80b0eec8e525b06ebd7eab708e14ece0182a729da6b814)

BibTeX

@misc{dc8d7371c8e91c,
  title = {GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations — arXiv:2610.22708},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22708},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8d7371c8e91c; sha256 b4dce5de19c1c1733e80b0eec8e525b06ebd7eab708e14ece0182a729da6b814},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22708. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Text-based world models have recently shown potential to simulate complex RL environment state transitions for a diverse set of use cases. However, no standardized benchmark exists to compare, study and improve these models. Through this paper, we attempt to solve this problem by building GlobeBench, a novel benchmark with 1,632 tasks across five separate domains and nineteen simulation skills. The tasks check whether models can predict the environment's response to a specific action given the full agent trajectory by understanding earlier state changes, following environment rules, and predicting the effects of actions on both related and unrelated objects. We compare the predicted environment response with an execution-derived reference, using a hybrid architecture with both deterministic gates and a semantic judge. Among 20 general-purpose LLMs and specialized text world models, GPT-6 Astra achieves the highest task success rate at 49.57%. We find that model accuracy decreases as actions affect a larger portion of the environment. Specifically, when the agent's changes must propagate to dependent objects, task success drops to an average of 5.1% across all 20 models. Although model strengths differ across tasks, 631 tasks remain unsolved by any model, suggesting a significant gap in general LLM capabilities as a world model.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.22708 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:10.027Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).