Metraipolis › archive › dc_cea8dcfa1d4f

TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding — arXiv:2609.07937

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.07937
License (as archived)
CC0-1.0
Archived
2026-10-16T15:11:59.974Z
SHA-256 of the archived text
d27f918ee018b60c5edcfba49891df3a99062539e3370a63fc09a521999a7273
Stable URL of this page
https://metraipolis.com/documento/dc_cea8dcfa1d4f

How to cite

TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding — arXiv:2609.07937 — arXiv (en), https://arxiv.org/abs/2609.07937 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cea8dcfa1d4f (Metraipolis archive dc_cea8dcfa1d4f, sha256 d27f918ee018b60c5edcfba49891df3a99062539e3370a63fc09a521999a7273)
BibTeX
@misc{dccea8dcfa1d4f,
  title = {TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding — arXiv:2609.07937},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.07937},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cea8dcfa1d4f; sha256 d27f918ee018b60c5edcfba49891df3a99062539e3370a63fc09a521999a7273},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.07937. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Structured visual reasoning, such as image puzzles, demands fine-grained visual perception, which current Vision Language Models (VLMs) often lack. VLMs built on CLIP-based ViT backbones tend to trade fine-grained visual details for high-level semantics. We show that this deficit propagates downstream, affecting various vision and language tasks. To address this, we fuse DINOv3 and CleanDIFT representations into a perception encoder (DiffusedDINO), and align it with RoBERTa-L, yielding a text-aligned model (TDDN) that preserves this perceptual advantage: with frozen backbones and only ~590K alignment pairs, TDDN surpasses CLIP ViT-L/14 in image-text retrieval tasks: text-to-image on Flickr30K and both directions on MS-COCO-2014. TDDN more than triples CLIP's dense-prediction accuracy (ADE20K 5.20 -> 18.11 mIoU, COCO-Stuff 7.35 -> 24.44), despite CLIP's massive training corpus. TDDN leads on segmentation benchmarks among general-purpose contrastive encoders, including SigLIP 2. We further introduce PuzzlePerception, a segmentation and visual question answering dataset that probes fine-grained spatial understanding, on which TDDN doubles CLIP's segmentation accuracy (11.04 -> 22.51 mIoU). Our code and data are available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.07937 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T14:49:57.258Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).