Metraipolis › archive › dc_4522f2ef2cd4

Describe Your Facial Expressions by Linking Image Encoders and Large Language Models — arXiv:2611.08138

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.08138
License (as archived): CC0-1.0
Archived: 2026-11-06T14:53:42.939Z
SHA-256 of the archived text: 91785cb310c55ea52322ac85a272d6e5ff631c4fef8e30880fd716c733bc02c5
Stable URL of this page: https://metraipolis.com/documento/dc_4522f2ef2cd4

How to cite

Describe Your Facial Expressions by Linking Image Encoders and Large Language Models — arXiv:2611.08138 — arXiv (en), https://arxiv.org/abs/2611.08138 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4522f2ef2cd4 (Metraipolis archive dc_4522f2ef2cd4, sha256 91785cb310c55ea52322ac85a272d6e5ff631c4fef8e30880fd716c733bc02c5)

BibTeX

@misc{dc4522f2ef2cd4,
  title = {Describe Your Facial Expressions by Linking Image Encoders and Large Language Models — arXiv:2611.08138},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08138},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4522f2ef2cd4; sha256 91785cb310c55ea52322ac85a272d6e5ff631c4fef8e30880fd716c733bc02c5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08138. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This paper presents a novel task of describing human facial expressions of a facial image in natural language, which captures the nuances of facial actions and emotional states beyond traditional emotion categories or facial action units (AUs). To achieve the facial expression captioning model, we propose a three-stage training framework that trains a vision-to-language model using synthesized image-text pairs and the BLIP-2 pre-training techniques. To overcome the challenge of missing training image-text pairs for facial expression captioning, we propose a strategy that involves synthesizing and combining captions using GPT-3.5 and existing annotations on either emotion categories or AUs. Experiments demonstrate the effectiveness of our method in generating captions that describe details of facial actions and emotions, as well as the inferential relationship between them, even when those emotions are not present in the training data. It is also demonstrated that the vision-to-language task enhances the performance of the intermediate visual features on both AU detection and emotion classification tasks.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.08138 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:25:42.626Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).