Metraipolis › archive › dc_f3baa15dfdc8
RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection — arXiv:2610.09502
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.09502
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T12:47:23.139Z
- SHA-256 of the archived text
62f79e61237ec2f969d959e951a13b0b8181559efd08f136c6e574e40d179cd7- Stable URL of this page
https://metraipolis.com/documento/dc_f3baa15dfdc8
How to cite
RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection — arXiv:2610.09502 — arXiv (en), https://arxiv.org/abs/2610.09502 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f3baa15dfdc8 (Metraipolis archive dc_f3baa15dfdc8, sha256 62f79e61237ec2f969d959e951a13b0b8181559efd08f136c6e574e40d179cd7)
BibTeX
@misc{dcf3baa15dfdc8,
title = {RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection — arXiv:2610.09502},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.09502},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f3baa15dfdc8; sha256 62f79e61237ec2f969d959e951a13b0b8181559efd08f136c6e574e40d179cd7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09502. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Open-vocabulary detection (OVD) recognizes categories unseen during training through textual category queries, yet achieving strong generalization with real-time efficiency remains challenging. Beyond vocabulary scaling, zero-shot generalization may benefit from reusable visual--semantic cues learned from seen data, including attributes, actions, states, and contextual relations. Existing real-time OVD methods primarily emphasize vocabulary coverage and efficient region/query--text matching; under strict efficiency constraints, compact detectors may struggle to absorb rich instance semantics and scene context. We propose RT-DETR-World, a compact DETR-style detector that transfers the rich semantics conveyed by descriptions during training while retaining lightweight query--text matching at inference. We construct GroundingCapv2 with three levels of supervision: category names for standard OVD, object descriptions conveying instance-level semantics, and image descriptions conveying object relations and scene context. These descriptions serve only as training-time semantic supervision. To help the compact detector absorb these semantics, we propose Dual-Path Description Alignment (DDA), combining a deployment-consistent MiniLM pathway with a training-only LLM teacher. MiniLM provides query--category supervision and object-description alignment, while offline teacher features supervise matched queries and global visual representations at the object and image levels, respectively. All teacher features are precomputed, and the teacher-side modules are removed after training. We further propose Relation-Aware Negative Relaxation (RNR), which uses teacher-derived semantic similarities to relax related negatives while preserving exact positives. Experiments demonstrate competitive zero-shot accuracy and a favorable accuracy--efficiency trade-off. The code will be released. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2610.09502 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T12:18:15.378Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).