Metraipolis › archive › dc_e1a54ee11697

A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Data — arXiv:2608.08663

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08663
License (as archived)
CC0-1.0
Archived
2026-10-19T08:59:35.767Z
SHA-256 of the archived text
c7491b6febc2e14b547e899ba4f7ddad7a7a91a37bc2dee4492e819cd74f1dc4
Stable URL of this page
https://metraipolis.com/documento/dc_e1a54ee11697

How to cite

A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Data — arXiv:2608.08663 — arXiv (en), https://arxiv.org/abs/2608.08663 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e1a54ee11697 (Metraipolis archive dc_e1a54ee11697, sha256 c7491b6febc2e14b547e899ba4f7ddad7a7a91a37bc2dee4492e819cd74f1dc4)
BibTeX
@misc{dce1a54ee11697,
  title = {A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Data — arXiv:2608.08663},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08663},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e1a54ee11697; sha256 c7491b6febc2e14b547e899ba4f7ddad7a7a91a37bc2dee4492e819cd74f1dc4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08663. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Humans converge on shared names for novel, hard-to-describe objects through repeated interaction, a process psycholinguists call lexical entrainment. Leading vision-language models fail at this: recent empirical work documents that they do not shorten references, reuse successful expressions, or maintain stable pact state across turns. We present a framework that addresses the gap by externalizing pact state into three explicit, inspectable sets of referent-object bindings ($\Gamma, \Xi, \Omega$), updated by a dynamic-semantics context-change rule. The symbolic layer sits on top of a lightweight perceptual-alignment pipeline that grounds noisy human referring expressions in crowd-sourced imagery via SIFT homographies and the Universal Quality Index. Evaluated on the Stanford Repeated Reference Game corpus (over 15{,}000 director-matcher utterances on abstract tangram stimuli), the framework places the correct target in its top-5 hypothesis set 83.56% of the time from a single director utterance. Human matcher top-1 accuracy on the same corpus is approximately 77-80%. We also report results on a held-out condition in which obvious tangram-adjacent images are excluded from the retrieved set, which provides a more conservative measurement of the grounding signal. Ablations isolate the contribution of each component: SIFT alignment, UQI, query preprocessing, and image augmentation. The central contribution is the combination: a transparent, auditable symbolic layer that recovers the structure of lexical entrainment turn by turn, paired with a perceptual channel whose behavior can be examined ablation by ablation. We also discuss in detail what the framework does not do. It is not interactive, it does not close the loop with the director, and its retrieval-driven perceptual channel is vulnerable to a class of leakage effects that we quantify and bound rather than wave away.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.HC).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.08663 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:54:56.005Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).