Metraipolis › archive › dc_eeffe95e02e5

Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions — arXiv:2610.41951

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41951
License (as archived): CC0-1.0
Archived: 2026-11-02T07:40:51.297Z
SHA-256 of the archived text: 9d06924e7241bf41b087a069f8d0d248b454ce1772db8ea35022ba44e0092f2f
Stable URL of this page: https://metraipolis.com/documento/dc_eeffe95e02e5

How to cite

Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions — arXiv:2610.41951 — arXiv (en), https://arxiv.org/abs/2610.41951 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eeffe95e02e5 (Metraipolis archive dc_eeffe95e02e5, sha256 9d06924e7241bf41b087a069f8d0d248b454ce1772db8ea35022ba44e0092f2f)

BibTeX

@misc{dceeffe95e02e5,
  title = {Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions — arXiv:2610.41951},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41951},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eeffe95e02e5; sha256 9d06924e7241bf41b087a069f8d0d248b454ce1772db8ea35022ba44e0092f2f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41951. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal Large Language Models (MLLMs) have achieved impressive performance across a wide range of reasoning tasks, yet it remains unclear whether their internal representations and decision criteria align with the latent structure of human cognition. In this work, we propose a novel framework to align MLLMs with the multidimensional mental representations underlying human similarity judgements. We build on a previously established behavior-derived semantic embedding space, in which objects are represented along 66 interpretable dimensions derived from 4.70 million human Odd-One-Out (O1O) judgements. For each triplet, we infer the most salient decision dimension and convert it into a dimension-grounded linguistic rationale. This enables MLLMs to learn both the final choice and a justification consistent with the inferred behavioral criterion. Experimental results demonstrate that our approach improves model consistency with human similarity judgements while retaining competitive performance on standard multimodal benchmarks. Furthermore, using searchlight Representational Similarity Analysis (RSA) on an independent fMRI Natural Scenes Dataset (NSD), we observe increased representational alignment between model and human. Ultimately, our findings point to a data-driven route for incorporating human cognitive structure into MLLMs alignment. The code is available \href{ this https URL }{here}.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41951 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:37:00.596Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).