Metraipolis › archive › dc_56a009c985bf

MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models — arXiv:2609.01772

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.01772
License (as archived)
CC0-1.0
Archived
2026-10-17T04:16:29.945Z
SHA-256 of the archived text
21dfad31badca7cc803959789246baa1d95d296d5439ec28b7f808a5bd5a1f9b
Stable URL of this page
https://metraipolis.com/documento/dc_56a009c985bf

How to cite

MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models — arXiv:2609.01772 — arXiv (en), https://arxiv.org/abs/2609.01772 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_56a009c985bf (Metraipolis archive dc_56a009c985bf, sha256 21dfad31badca7cc803959789246baa1d95d296d5439ec28b7f808a5bd5a1f9b)
BibTeX
@misc{dc56a009c985bf,
  title = {MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models — arXiv:2609.01772},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01772},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_56a009c985bf; sha256 21dfad31badca7cc803959789246baa1d95d296d5439ec28b7f808a5bd5a1f9b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01772. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vision-language models still lack. We introduce MemeCULT-1K, a multilingual benchmark of 1,000 South Asian memes in Bengali, English, and Hindi, where each meme is paired with a cultural context note and three human-written explanations, along with a supplementary set of 54 Bengali regional dialect memes. We evaluate thirteen popular Vision Language Models (VLMs) under two settings: meme-only and context-aware. Providing minimal cultural context yields consistent gains across all models and languages: mean SBERT similarity improves from 44.6 to 56.4 (+11.8), BLEURT from 37.3 to 42.3 (+5.0), and LLM-as-a-Judge scores from 2.57 to 3.43 out of 5 (+0.86). Fine-grained error analysis reveals that closed-source models fail mainly on entity and reference misidentification, while open-source models are bottlenecked by broader cultural knowledge gaps, with linguistic and phonological failures proving the most context-resistant across both. These results highlight the difficulty of culturally grounded meme understanding and motivate future work on explicit cultural knowledge integration. Our dataset and code are publicly available at TawsifDipto17/MemeCULT-1K.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.01772 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:26:49.757Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).