Metraipolis › archive › dc_803726087ef8

Surrogate Uncertainty for Black-box Visual Grounding — arXiv:2610.39798

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39798
License (as archived)
CC0-1.0
Archived
2026-11-01T02:45:42.519Z
SHA-256 of the archived text
61fe6e8d7cb5e9488a447941f74c568af1ab8de83f7bca95ff5c14755570570a
Stable URL of this page
https://metraipolis.com/documento/dc_803726087ef8

How to cite

Surrogate Uncertainty for Black-box Visual Grounding — arXiv:2610.39798 — arXiv (en), https://arxiv.org/abs/2610.39798 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_803726087ef8 (Metraipolis archive dc_803726087ef8, sha256 61fe6e8d7cb5e9488a447941f74c568af1ab8de83f7bca95ff5c14755570570a)
BibTeX
@misc{dc803726087ef8,
  title = {Surrogate Uncertainty for Black-box Visual Grounding — arXiv:2610.39798},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39798},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_803726087ef8; sha256 61fe6e8d7cb5e9488a447941f74c568af1ab8de83f7bca95ff5c14755570570a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39798. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visual grounding is crucial for robots to understand language instructions with visual observations. This paper addresses the problem of visual grounding uncertainties, where a robot uses a black-box vision-language model (VLM) to ground free-form natural-language instructions to target objects. Prevailing methods derive uncertainty from the VLM itself. Yet, outputs of the VLM, e.g., generated uncertainties, are from the same parameters and structures producing the predictions, and inherit the bias. We give a formal Bayesian definition of the problem that treats the VLM prediction as an observation. An external surrogate Bayesian model then quantifies per-query uncertainty over a local neighborhood of a held-out calibration set, decomposed into aleatoric and epistemic components. A hierarchical retrieval across input modalities realizes a compact joint-space neighborhood, mitigating the curse of dimensionality. The estimator further integrates with conformal prediction or model-based planning for active visual grounding, where robots are allowed to take verbal or physical actions to ground targets. Across grounding benchmarks and state-of-the-art VLMs, it consistently improves both calibration and task performance. On a real Fetch robot under open-world instructions, it enables principled trade-offs between confident action and information gathering, substantially improving robustness.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.RO, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.39798 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:16:58.670Z dalla pagina https://arxiv.org/list/cs.RO/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).