Metraipolis › archive › dc_b4396ef97135

GeoGuesser Bench: Benchmarking Vision-Language Agents on Interactive Geolocation — arXiv:2611.02698

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02698
License (as archived)
CC0-1.0
Archived
2026-11-03T10:53:42.051Z
SHA-256 of the archived text
92c42724b6a2f9f05c3fab2cfdcd08d08e0aac426c854b5391abe14f1c36b311
Stable URL of this page
https://metraipolis.com/documento/dc_b4396ef97135

How to cite

GeoGuesser Bench: Benchmarking Vision-Language Agents on Interactive Geolocation — arXiv:2611.02698 — arXiv (en), https://arxiv.org/abs/2611.02698 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b4396ef97135 (Metraipolis archive dc_b4396ef97135, sha256 92c42724b6a2f9f05c3fab2cfdcd08d08e0aac426c854b5391abe14f1c36b311)
BibTeX
@misc{dcb4396ef97135,
  title = {GeoGuesser Bench: Benchmarking Vision-Language Agents on Interactive Geolocation — arXiv:2611.02698},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02698},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b4396ef97135; sha256 92c42724b6a2f9f05c3fab2cfdcd08d08e0aac426c854b5391abe14f1c36b311},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02698. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language model (VLM) agents that act in the open world need two abilities: world knowledge to interpret what they see, and exploration to find what they cannot yet see. We study how agents combine the two through interactive visual geolocation. GeoGuesser Bench contains 282 tasks in an offline Street View environment where agents rotate, zoom, and move between real panoramas. Its three task types shift the balance from knowledge to exploration: agents (1) report the coordinates of their starting location, (2) name the enclosing street, city, state, or country, or (3) find, read, and count nearby objects. The coordinate tasks in (1) also include 42 real GeoGuessr World Championship rounds, whose expert guesses provide a human reference. We evaluate 23 VLMs: the best accuracy is 62.5% on place naming and 35.0% on object finding, and the best agent averages 4352.4 GeoGuessr points on the championship rounds, against 4770.3 for the better expert in each round. Beyond these headline numbers, world knowledge and exploration are only weakly correlated across models: agents that localize well do not necessarily explore well. To understand where agents fail, we categorize their errors into vision, navigation, and tool-use failures. Further ablations show that the strongest agents, such as Claude Opus 4.8, still perform well with limited views, while weaker agents degrade; customizing the environment thus adjusts task difficulty. The environment's verifiable rewards and restorable state also support training, and our Planner-Worker-Compressor harness cuts planner tokens by 57-61% by compacting the interaction history. We hope GeoGuesser Bench and its environment serve as a high-quality foundation for simulating real-world exploration at scale.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.02698 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.378Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).