Metraipolis › archive › dc_ecfb94299351
GEB-Bench: Abstract Structures Told in Many Voices — arXiv:2608.04111
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04111
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:25:28.721Z
- SHA-256 of the archived text
bd2b928f3b3cd5e41b6b0fe52bc91e94d814a58ecab350ba9ecacecc57384066- Stable URL of this page
https://metraipolis.com/documento/dc_ecfb94299351
How to cite
GEB-Bench: Abstract Structures Told in Many Voices — arXiv:2608.04111 — arXiv (en), https://arxiv.org/abs/2608.04111 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ecfb94299351 (Metraipolis archive dc_ecfb94299351, sha256 bd2b928f3b3cd5e41b6b0fe52bc91e94d814a58ecab350ba9ecacecc57384066)
BibTeX
@misc{dcecfb94299351,
title = {GEB-Bench: Abstract Structures Told in Many Voices — arXiv:2608.04111},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04111},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ecfb94299351; sha256 bd2b928f3b3cd5e41b6b0fe52bc91e94d814a58ecab350ba9ecacecc57384066},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04111. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Analogy allows models to apply relational knowledge from one situation to another. Benchmarks of equivalent representations test different ways of expressing the same underlying problem, whereas analogy often links situations involving different objects and content. To evaluate the analogy ability of models, we introduce GEB-Bench, a benchmark centered on cross-voice motif matching. A voice presents a motif as a scene, story, mathematical statement, or programmatic skeleton. The first task is identification across these voices. The second is matching scenes to stories or mathematical statements, and stories to mathematical statements, by their shared motif. A complete construction pipeline generates items in these voices from the relation specified for each motif, checks the order and repetition of story units, and assembles paired questions. We evaluate current models on our benchmark. Results show that matching performance depends on the target voice. The paired scene-to-story study reveals that (1) evaluated models still make matching errors when they separately identify the source and every candidate correctly, and (2) identification improves more consistently than scene-to-story matching across the evaluated Qwen model sizes. These findings show that successful motif identification does not ensure reliable matching across voices. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CL). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL); Logic in Computer Science (cs.LO). Pagina del lavoro: https://arxiv.org/abs/2608.04111 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:13:23.205Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).