InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark

2026-06-01

AI Summary

This work addresses the limitations of existing visual emotion understanding benchmarks, which predominantly focus on emotion recognition while neglecting the assessment of concrete affective grounding and cognitive reasoning capabilities. To bridge this gap, we introduce InsightVQA-Bench—a large-scale visual question answering benchmark structured around three hierarchical levels: perception, grounded affective understanding, and cognitive reasoning—uniquely integrating emotion recognition, visual trigger attribution, and intention prediction. Leveraging multi-stage image filtering and constraint-guided question-answer generation, we construct a dataset comprising 138K images and 725K question-answer pairs, along with a fine-grained evaluation subset of 30K samples. We further develop InsightNet, a multimodal large language model baseline fine-tuned for emotion understanding. Experimental results demonstrate that the proposed benchmark poses significant challenges to current models, effectively advancing visual emotion understanding toward higher-order cognitive reasoning.

Abstract

Visual emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive reasoning. However, existing benchmarks mainly focus on emotion recognition, offering limited support for grounded understanding and response-oriented analysis. To address this gap, we introduce \textbf{InsightVQA}, a large-scale dataset for hierarchical visual question answering on emotion understanding and cognitive reasoning. Building from 351K images collected from six public sources, we apply a rigorous multi-stage filtering pipeline to curate 138K high-confidence images. Each image is annotated at three hierarchical levels: perception QA for emotion and valence recognition, grounded understanding QA constructed from visual trigger extraction through constraint-guided generation, and cognition QA centered on response intent prediction and sequential insight reasoning. In total, InsightVQA contains 725K QA pairs. We further present \textbf{InsightVQA-Bench}, a high-quality evaluation benchmark comprising 30K samples for fine-grained evaluation. To support evaluation, we introduce \textbf{InsightNet}, an emotion-tuned baseline for MLLMs. Results demonstrate that InsightVQA poses significant challenges for grounded emotion understanding and reasoning.

Problem

• Visual Question Answering
• Emotion Understanding
• Cognitive Reasoning
• Grounded Understanding
• Hierarchical QA

Innovation

• Emotion-Cognitive Reasoning
• Hierarchical Visual Question Answering
• Grounded Understanding
• Constraint-Guided Generation
• Multimodal Large Language Models