Metraipolis › archive › dc_3fb60faf4c8d

From "What-If" to "What-Is": Counterfactual Thinking-Inspired Semantic Alignment for Visual Brain Decoding — arXiv:2608.15163

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15163
License (as archived)
CC0-1.0
Archived
2026-10-18T19:39:31.574Z
SHA-256 of the archived text
d3ba2990ef5a13688b14f268af267c6a2155d7de2d5b7e3aa28637733734fcc4
Stable URL of this page
https://metraipolis.com/documento/dc_3fb60faf4c8d

How to cite

From "What-If" to "What-Is": Counterfactual Thinking-Inspired Semantic Alignment for Visual Brain Decoding — arXiv:2608.15163 — arXiv (en), https://arxiv.org/abs/2608.15163 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3fb60faf4c8d (Metraipolis archive dc_3fb60faf4c8d, sha256 d3ba2990ef5a13688b14f268af267c6a2155d7de2d5b7e3aa28637733734fcc4)
BibTeX
@misc{dc3fb60faf4c8d,
  title = {From "What-If" to "What-Is": Counterfactual Thinking-Inspired Semantic Alignment for Visual Brain Decoding — arXiv:2608.15163},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15163},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3fb60faf4c8d; sha256 d3ba2990ef5a13688b14f268af267c6a2155d7de2d5b7e3aa28637733734fcc4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15163. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visual brain decoding reconstructs visual content perceived by a person from neural measurements such as fMRI, providing a computational approach to studying how visual information is represented in the brain. Recent multimodal representations and diffusion priors have improved reconstruction realism. However, visually plausible reconstructions may contain incorrect objects, attributes, or relations because a strong generative prior can complete content not sufficiently specified by the decoded representation. Conventional reconstruction metrics mainly assess the final image and may therefore obscure such semantic errors. We propose ConceptAlign, a counterfactual semantic alignment framework for visual brain decoding. ConceptAlign pools decoded visual tokens and projects them into a frozen text-embedding space, aligning the representation with the ground-truth caption while separating it from scene-preserving near-miss alternatives. Generated offline by an LLM, these alternatives modify one critical object, attribute, or relation while retaining the scene. A margin-based objective learns fine-grained semantic boundaries between the observed stimulus and plausible but incorrect interpretations without requiring LLM calls during inference. We introduce a systematic three-level semantic evaluation framework covering foundational discriminability, counterfactual description discrimination, and representational geometry. Experiments on the Natural Scenes Dataset show that ConceptAlign improves reconstruction measures, counterfactual semantic discrimination, and representational alignment over the MindEye2 backbone. Matched negative-source ablations, independent LLM and human-written alternatives, and human evaluation support the effectiveness and robustness of the supervision, with favorable patterns in fine-grained conflicts, limited-data decoding, and cross-subject structure.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.HC).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2608.15163 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T19:02:17.226Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).