Metraipolis › archive › dc_0696ed82dbc4
Witness: Discovery, Deciphering, and Epiphany in Interactive Puzzle Environments — arXiv:2609.32208
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32208
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:55:11.759Z
- SHA-256 of the archived text
205a0b08681355e285a7814f2d4b4e9e1aefe166ae1becac6becc2480592a5e2- Stable URL of this page
https://metraipolis.com/documento/dc_0696ed82dbc4
How to cite
Witness: Discovery, Deciphering, and Epiphany in Interactive Puzzle Environments — arXiv:2609.32208 — arXiv (en), https://arxiv.org/abs/2609.32208 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0696ed82dbc4 (Metraipolis archive dc_0696ed82dbc4, sha256 205a0b08681355e285a7814f2d4b4e9e1aefe166ae1becac6becc2480592a5e2)
BibTeX
@misc{dc0696ed82dbc4,
title = {Witness: Discovery, Deciphering, and Epiphany in Interactive Puzzle Environments — arXiv:2609.32208},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32208},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0696ed82dbc4; sha256 205a0b08681355e285a7814f2d4b4e9e1aefe166ae1becac6becc2480592a5e2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32208. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automated science needs agents that can work out the rules of an unfamiliar environment by interacting with it. Interactive rule-discovery puzzles offer a controlled setting for studying this ability: an agent infers hidden rules through experimentation and uses what it has inferred to reach a stated goal. We ask what limits current language models on these puzzles and whether reinforcement learning (RL) improves performance on rules held out from training. To study both, we introduce WITNESS, a 2D grid-based puzzle environment with ground-truth ASCII observations and controlled access to rules. An agentic pipeline generates games for WitnessGym, the RL training suite, and WitnessBench, comprising public validation and private test games. The validation set separately tests new compositions of trained rule primitives and primitives absent from training. Under a shared harness, the best of 18 frontier proprietary and open-weight models solves only 24\% of private test level slots, with scores sensitive to the observation interface and agent configuration. Providing ground-truth rules raises Opus-5's validation RHAE-L5 (relative human action efficiency over the first five levels) from 59.9 to 97.8, whereas a 27B open-weight model gains only 2.1 points and remains limited even with the rules provided. RL on WitnessGym raises the 27B model's private test RHAE-L5 from 2.1 to 5.4 and yields a mean gain of 4.1 points on four external discovery benchmarks. Together, these results point to rule acquisition as a major difficulty for frontier models like Opus-5 while smaller models further struggle on rule-based execution, and indicate that RL on hidden-rule puzzles transfers to broader rules and real-world tasks beyond training. Benchmark is available at: this https URL Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.32208 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:00.728Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).