Metraipolis › archive › dc_15db89b4099d
Talking to Me or Someone Else? Rethinking Talk-to-Me Detection in Egocentric Videos — arXiv:2609.14118
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14118
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:50:53.679Z
- SHA-256 of the archived text
6096d2236c1b294bcebbf78e42dd27656abb7ecd4d0d11e1fd14d29c8e846bc6- Stable URL of this page
https://metraipolis.com/documento/dc_15db89b4099d
How to cite
Talking to Me or Someone Else? Rethinking Talk-to-Me Detection in Egocentric Videos — arXiv:2609.14118 — arXiv (en), https://arxiv.org/abs/2609.14118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_15db89b4099d (Metraipolis archive dc_15db89b4099d, sha256 6096d2236c1b294bcebbf78e42dd27656abb7ecd4d0d11e1fd14d29c8e846bc6)
BibTeX
@misc{dc15db89b4099d,
title = {Talking to Me or Someone Else? Rethinking Talk-to-Me Detection in Egocentric Videos — arXiv:2609.14118},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14118},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_15db89b4099d; sha256 6096d2236c1b294bcebbf78e42dd27656abb7ecd4d0d11e1fd14d29c8e846bc6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Online understanding of who is talking to the camera wearer is a key capability for egocentric social interaction. However, existing talk-to-me (TTM) studies are commonly formulated as offline clip-level recognition, which is poorly aligned with online interaction and overlooks the diverse non-TTM speaking states that naturally arise in egocentric videos. In this paper, we revisit this problem by reformulating it as an online, frame-level prediction task. Instead of treating TTM as a binary problem against a single negative class, we model it in the presence of diverse and previously underexplored non-TTM states, such as talking-to-others, self-talking, and background conditions. To support this new formulation, we construct an Online TTM Dataset consisting of 406 egocentric video clips with approximately 900K annotated frames, each labeled with frame-level social interaction categories (e.g., background, TTM, talking-to-others, self-talking), by extending the Ego4D social interaction benchmark. In this benchmark, we evaluate five adapted baselines and develop a new model that integrates social cues across modalities. Experimental results show that our multimodal model, which jointly leverages audio, visual, and speech-semantic cues, achieves 75.5% frame-level F1 on TTM, outperforming strong baselines and enabling a systematic analysis of how different speaking states affect TTM recognition. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.14118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.623Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).