Metraipolis › archive › dc_458b49deddee

Egocentric Action Recognition with Objects and Vision-Language Affordances — arXiv:2611.07673

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.07673
License (as archived): CC0-1.0
Archived: 2026-11-06T14:53:29.780Z
SHA-256 of the archived text: b43482f636de92b45ee8b684e617bacb36d6c91331a26bdd35f730e8380dc75c
Stable URL of this page: https://metraipolis.com/documento/dc_458b49deddee

How to cite

Egocentric Action Recognition with Objects and Vision-Language Affordances — arXiv:2611.07673 — arXiv (en), https://arxiv.org/abs/2611.07673 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_458b49deddee (Metraipolis archive dc_458b49deddee, sha256 b43482f636de92b45ee8b684e617bacb36d6c91331a26bdd35f730e8380dc75c)

BibTeX

@misc{dc458b49deddee,
  title = {Egocentric Action Recognition with Objects and Vision-Language Affordances — arXiv:2611.07673},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07673},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_458b49deddee; sha256 b43482f636de92b45ee8b684e617bacb36d6c91331a26bdd35f730e8380dc75c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07673. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Action recognition, and more specifically egocentric action recognition, is a core task in computer vision, with applications such as monitoring the daily life activities of elderly people and task assistance. Recent methods rely on visual modalities such as RGB video or depth. However, in egocentric video, an action is largely defined by its objects, which these methods rarely model. We propose a method that combines a frozen SlowFast video expert with an attention-based object expert by late fusion. Each object is represented with its affordances, that is, what can be done with it, generated zero-shot by a vision-language model. Since datasets such as MECCANO annotate only the objects involved in each action, evaluating with these boxes tells the model which objects are active. We therefore train on annotated boxes and evaluate the method with detections from the official active-object detector, and with annotated boxes as an upper bound. Without affordances, our fusion reaches 68.4% Top-1 with annotated boxes but 49.0% with detections. About half of this drop comes from non-active detections. To remove them, the same vision-language model generates an interaction affordance that decides whether a hand is manipulating each detected object. On MECCANO, objects raise the Top-1 of the video expert from 44.9% to 49.0%, and the two affordances raise it to 49.9%, above SlowFast and object-aware RGB baselines. Code will be released upon acceptance.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.07673 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:25:42.615Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).