Cause-Effect Driven Optimization for Robust Medical Visual Question Answering with Language Biases

2025-06-22

AI Summary

Medical visual question answering (Med-VQA) models suffer from language bias—spurious correlations between question types and answer categories induced by statistical shortcuts in imbalanced training data—undermining cross-modal reasoning robustness. To address this, we propose CEDO, a causal-driven debiasing framework that jointly models the *causal origin* (data imbalance) and *causal effect* (shortcut learning) of bias. CEDO introduces three core components: (i) modality-heterogeneous optimization, (ii) gradient-guided modality coordination, and (iii) distribution-adaptive loss rescaling. Further, it integrates Pareto-optimal multimodal fusion, gradient orthogonalization constraints, and dynamic multi-task loss weighting for end-to-end debiasing. Extensive experiments on multiple standard and bias-sensitive Med-VQA benchmarks demonstrate that CEDO significantly outperforms state-of-the-art methods, achieving superior generalization and training stability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Existing Medical Visual Question Answering (Med-VQA) models often suffer from language biases, where spurious correlations between question types and answer categories are inadvertently established. To address these issues, we propose a novel Cause-Effect Driven Optimization framework called CEDO, that incorporates three well-established mechanisms, i.e., Modality-driven Heterogeneous Optimization (MHO), Gradient-guided Modality Synergy (GMS), and Distribution-adapted Loss Rescaling (DLR), for comprehensively mitigating language biases from both causal and effectual perspectives. Specifically, MHO employs adaptive learning rates for specific modalities to achieve heterogeneous optimization, thus enhancing robust reasoning capabilities. Additionally, GMS leverages the Pareto optimization method to foster synergistic interactions between modalities and enforce gradient orthogonality to eliminate bias updates, thereby mitigating language biases from the effect side, i.e., shortcut bias. Furthermore, DLR is designed to assign adaptive weights to individual losses to ensure balanced learning across all answer categories, effectively alleviating language biases from the cause side, i.e., imbalance biases within datasets. Extensive experiments on multiple traditional and bias-sensitive benchmarks consistently demonstrate the robustness of CEDO over state-of-the-art competitors.

Problem

• Addresses language biases in Medical Visual Question Answering models
• Proposes CEDO framework to mitigate biases from causal and effectual perspectives
• Enhances robustness via modality optimization and balanced loss rescaling

Innovation

• Modality-driven Heterogeneous Optimization for robust reasoning
• Gradient-guided Modality Synergy to eliminate bias updates
• Distribution-adapted Loss Rescaling for balanced learning