MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models

2026-09-01

AI Summary

本文通过构建包含1000个南亚模因的多语言基准MemeCULT-1K，旨在评估和改善视觉-语言模型在理解含有文化背景信息的模因方面的能力。

Abstract

Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vision-language models still lack. We introduce MemeCULT-1K, a multilingual benchmark of 1,000 South Asian memes in Bengali, English, and Hindi, where each meme is paired with a cultural context note and three human-written explanations, along with a supplementary set of 54 Bengali regional dialect memes. We evaluate thirteen popular Vision Language Models (VLMs) under two settings: meme-only and context-aware. Providing minimal cultural context yields consistent gains across all models and languages: mean SBERT similarity improves from 44.6 to 56.4 (+11.8), BLEURT from 37.3 to 42.3 (+5.0), and LLM-as-a-Judge scores from 2.57 to 3.43 out of 5 (+0.86). Fine-grained error analysis reveals that closed-source models fail mainly on entity and reference misidentification, while open-source models are bottlenecked by broader cultural knowledge gaps, with linguistic and phonological failures proving the most context-resistant across both. These results highlight the difficulty of culturally grounded meme understanding and motivate future work on explicit cultural knowledge integration. Our dataset and code are publicly available at TawsifDipto17/MemeCULT-1K.

Problem

• Meme Understanding
• Cultural Context
• Vision-Language Models
• South Asian

Innovation

• multilingual benchmark
• cultural context
• vision-language models
• meme understanding