Metraipolis › archive › dc_2d2225616e4d

CulturalCrosswords: A Multilingual Benchmark comprised of Answer Constraints and Grid Completion — arXiv:2611.10044

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10044
License (as archived)
CC0-1.0
Archived
2026-11-10T18:42:13.760Z
SHA-256 of the archived text
40358b82d10f9d8640f8f7f527747427f4c4bfc1e8ccaa692d1e42e2caf39a98
Stable URL of this page
https://metraipolis.com/documento/dc_2d2225616e4d

How to cite

CulturalCrosswords: A Multilingual Benchmark comprised of Answer Constraints and Grid Completion — arXiv:2611.10044 — arXiv (en), https://arxiv.org/abs/2611.10044 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d2225616e4d (Metraipolis archive dc_2d2225616e4d, sha256 40358b82d10f9d8640f8f7f527747427f4c4bfc1e8ccaa692d1e42e2caf39a98)
BibTeX
@misc{dc2d2225616e4d,
  title = {CulturalCrosswords: A Multilingual Benchmark comprised of Answer Constraints and Grid Completion — arXiv:2611.10044},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10044},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d2225616e4d; sha256 40358b82d10f9d8640f8f7f527747427f4c4bfc1e8ccaa692d1e42e2caf39a98},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10044. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When several answers fit a riddle, exact-match scoring can penalise a valid response. CulturalCrosswords tests how answer constraints affect large language models (LLMs) solving cultural riddles, whose solutions draw on knowledge specific to a community. We constructed 53 crosswords from crowdsourced riddles in 42 languages and evaluated 11 LLM configurations on five versions of each clue set: plain batches, batches with lengths, batches with lengths and partial-answer hints, and crosswords requesting an answer list or a completed grid. On 962 clues with alternative answers, lengths improve agreement with the reference by 22.8 percentage points, compared with 8.4 points when accepted alternatives also receive credit. Part of the gain therefore appears to come from selecting a required answer form. Crossword input increases the share of all crossings with compatible answers by 9.7 points; requiring a completed grid reveals large differences between models. The benchmark distinguishes agreement with a reference from the ability to produce answers that fit together and occupy the correct cells.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10044 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.573Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).