Metraipolis › archive › dc_8706fda9f5ea

MMCoT$^3$: Empowering 3D Scene Understanding with 3D-Text Interleaved Multi-Modal Chain-of-Thought Reasoning — arXiv:2611.01676

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01676
License (as archived)
CC0-1.0
Archived
2026-11-03T18:54:36.649Z
SHA-256 of the archived text
0c4a460a784c7da93d2f350dae92e0345284add2521d95885d7d7175cb8a5368
Stable URL of this page
https://metraipolis.com/documento/dc_8706fda9f5ea

How to cite

MMCoT$^3$: Empowering 3D Scene Understanding with 3D-Text Interleaved Multi-Modal Chain-of-Thought Reasoning — arXiv:2611.01676 — arXiv (en), https://arxiv.org/abs/2611.01676 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8706fda9f5ea (Metraipolis archive dc_8706fda9f5ea, sha256 0c4a460a784c7da93d2f350dae92e0345284add2521d95885d7d7175cb8a5368)
BibTeX
@misc{dc8706fda9f5ea,
  title = {MMCoT$^3$: Empowering 3D Scene Understanding with 3D-Text Interleaved Multi-Modal Chain-of-Thought Reasoning — arXiv:2611.01676},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01676},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8706fda9f5ea; sha256 0c4a460a784c7da93d2f350dae92e0345284add2521d95885d7d7175cb8a5368},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01676. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Understanding and reasoning over 3D scenes from point clouds is essential for a broad range of AI applications. Recent 3D large language models have shown promising results, yet they still face two fundamental limitations: (i) insufficient capability for grounded multi-step reasoning in complex 3D scenes, and (ii) a text-centric inference paradigm that lacks effective interaction between textual reasoning and visual evidence. As a result, these models often fail to verify intermediate inferences, disambiguate similar object instances, and avoid hallucinations. To address these limitations, we propose MMCoT$^3$, the first 3D-Text Interleaved Chain-of-Thought (CoT) framework for 3D scene understanding. The core idea is to explicitly interleave textual reasoning with task-relevant 3D evidence, enabling the model to consult scene observations throughout the reasoning process. In addition, since a textual mention in 3D scenes may correspond to multiple candidate objects, naive object-mention alignment can inject incorrect evidence and mislead subsequent reasoning. To mitigate this multi-instance ambiguity, we design an ambiguity-aware evidence aggregation mechanism that uses query-guided cross-attention to fuse candidate object tokens and inject both evidence and ambiguity cues into the reasoning process. To better equip the model with multi-step reasoning capability, we further develop PointCoT, a large-scale 3D-specific CoT instruction tuning dataset with approximately 176.2K instances spanning captioning, question answering, dialogue, and planning tasks. Extensive experiments on six benchmarks show that our method consistently improves performance and achieves state-of-the-art results, validating the effectiveness of our proposed ambiguity-aware 3D-text interleaved reasoning for 3D scene understanding.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.01676 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:16:52.945Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).