Metraipolis › archive › dc_7c4137fc3ee1

SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction — arXiv:2608.27461

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.27461
License (as archived)
CC0-1.0
Archived
2026-10-17T10:47:43.033Z
SHA-256 of the archived text
8d1eaa8db380482619e4e221e4ae4bc2c3ecb5fdad188e95f32b36c6e2bcbeb0
Stable URL of this page
https://metraipolis.com/documento/dc_7c4137fc3ee1

How to cite

SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction — arXiv:2608.27461 — arXiv (en), https://arxiv.org/abs/2608.27461 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7c4137fc3ee1 (Metraipolis archive dc_7c4137fc3ee1, sha256 8d1eaa8db380482619e4e221e4ae4bc2c3ecb5fdad188e95f32b36c6e2bcbeb0)
BibTeX
@misc{dc7c4137fc3ee1,
  title = {SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction — arXiv:2608.27461},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27461},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7c4137fc3ee1; sha256 8d1eaa8db380482619e4e221e4ae4bc2c3ecb5fdad188e95f32b36c6e2bcbeb0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27461. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Relational reasoning requires the process of perceptual understanding, comparing, and integrating the underlying relationships between concepts. This ability consists of multiple categories, such as analogical, structural, and cause-effect, each capturing a different aspect of higher-order understanding. To examine the performance of multimodal large language models (MLLM) on these relational inference tasks, we developed SciReC, a model-adaptive multimodal academic dialog benchmark. As the relational reasoning process involves multiple representations and various factors (visual understanding, exhibiting knowledge, and memory recall), we propose DMRA, a deficit-based diagnostic framework that quantifies the contribution of these components to identify the primary cause of unsuccessful cases. Claude 4.6 achieved the best performance on the overall relational score with 73\%, followed by GPT 5.4 with 68\%. Performance trends indicate that open-source models achieve their lowest scores on spatial relations, while proprietary models struggle more with hierarchical and sequential relations. Across domains, model performance is lowest on Astronomy and highest on Psychology. The results of DMRA reveal that relational reasoning is the primary source of error across all models, followed by memory limitations.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.27461 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T10:33:42.436Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).