Counterfactual Reasoning for Robust Visual Question Answering

2026-09-14

AI Summary

本文针对视觉问答模型因语言偏差导致的分布外泛化能力差问题，提出了一种新的反事实对比学习框架，通过三个关键改进来提高模型性能。

Abstract

Modern Visual Question Answering (VQA) models often exploit spurious correlations in training data, leading to poor out-of-distribution (OOD) generalization due to language bias. Although counterfactual learning has shown promise, existing methods can be improved to better guide attention toward causal evidence and strengthen feature discrimination. To address this, we propose a novel training framework that enhances counterfactual contrastive learning for VQA. Our framework introduces three key contributions: (1) a three-stage curriculum for stable multi-objective optimization, (2) an enhanced Batch-Contrastive loss for more discriminative feature learning, and (3) two novel regularizers, Answer-Contrastive (AC) loss to refine the prediction space and Gradient-Discrepancy (GD) loss to enforce causal visual grounding. Our model achieves a competitive accuracy of 61.64% on the bias-sensitive VQA-CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, yielding a small generalization gap of 1.16%. This demonstrates a strong balance between OOD robustness and in-distribution performance.

Problem

• Visual Question Answering
• spurious correlations
• out-of-distribution generalization
• language bias

Innovation

• counterfactual contrastive learning
• multi-objective optimization
• Batch-Contrastive loss
• Answer-Contrastive loss
• Gradient-Discrepancy loss