Metraipolis › archive › dc_ccc8d5abfb9d

Bouba Beats Kiki, and Other Regularities in the Subjective Preferences of LLMs — arXiv:2610.37692

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.37692
License (as archived)
CC0-1.0
Archived
2026-10-31T06:10:59.947Z
SHA-256 of the archived text
478d62f1a130675a67a297f16f5254058cd9793cc2335e60d212178ede7e6e25
Stable URL of this page
https://metraipolis.com/documento/dc_ccc8d5abfb9d

How to cite

Bouba Beats Kiki, and Other Regularities in the Subjective Preferences of LLMs — arXiv:2610.37692 — arXiv (en), https://arxiv.org/abs/2610.37692 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ccc8d5abfb9d (Metraipolis archive dc_ccc8d5abfb9d, sha256 478d62f1a130675a67a297f16f5254058cd9793cc2335e60d212178ede7e6e25)
BibTeX
@misc{dcccc8d5abfb9d,
  title = {Bouba Beats Kiki, and Other Regularities in the Subjective Preferences of LLMs — arXiv:2610.37692},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.37692},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ccc8d5abfb9d; sha256 478d62f1a130675a67a297f16f5254058cd9793cc2335e60d212178ede7e6e25},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.37692. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Users engage with large language models on tasks that are purely objective ("When was Netflix founded?"), purely subjective ("What should I name my child?") or anywhere in-between. We have rich tools for evaluating model accuracy with respect to objective criteria. However, when it comes to subjective aspects of a task, we do not have similarly developed methodology at hand. In this paper, we take some early steps towards framing the question of subjective evaluation. Rather than measure how often a model is right or wrong on a question, we instead ask, how do models make choices among multiple viable responses without a clear winner? To study this question, we restrict our focus to short-form content, and consider comparison tasks among three such content types: numbers, baby names, and fictional cities. We identify 29 aspects of these short-form examples covering many areas of syntax and semantics (length, alphabetic order, popularity, vowel mix, spoken sound, etc.). We show first that models tend to have strong and reproducible preferences for or against some of these aspects. Second, we show that variations in how the model is asked to express a preference may cause some of these aspects to activate or deactivate in the model's selection. Third, we show that overall preference distributions among pairs may vary radically based on seemingly-unimportant variations in the nature of the question posed to the model, raising new questions about manipulation. And finally, we show that, if models are used both to create lists of alternatives and to rank them, there are unexpected interactions between the two stages that may also drive the models' stated preferences.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.37692 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.585Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).