Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation

2025-08-11

AI Summary

Existing methods for improving model robustness against spurious correlations typically rely on auxiliary group or spurious feature annotations and assume identical group sets across source and target domains, severely limiting practical applicability. Method: We propose a superclass-guided representation disentanglement framework that requires no source-domain group annotations. It leverages pre-trained vision-language models to uncover hierarchical semantic structures among class labels, employs gradient-based attention to separate superclass-relevant and -irrelevant features, and incorporates reinforcement learning to optimize the disentanglement process. Contribution/Results: Our approach eliminates dependence on explicit group supervision, enabling more flexible modeling of complex spurious associations and improved domain generalization. Extensive experiments on multiple benchmarks demonstrate significant improvements over state-of-the-art methods—both in quantitative metrics (e.g., worst-group accuracy) and qualitative visualizations confirming effective feature disentanglement.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Representation Learning
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

To enhance group robustness to spurious correlations, prior work often relies on auxiliary annotations for groups or spurious features and assumes identical sets of groups across source and target domains. These two requirements are both unnatural and impractical in real-world settings. To overcome these limitations, we propose a method that leverages the semantic structure inherent in class labels--specifically, superclass information--to naturally reduce reliance on spurious features. Our model employs gradient-based attention guided by a pre-trained vision-language model to disentangle superclass-relevant and irrelevant features. Then, by promoting the use of all superclass-relevant features for prediction, our approach achieves robustness to more complex spurious correlations without the need to annotate any source samples. Experiments across diverse datasets demonstrate that our method significantly outperforms baselines in domain generalization tasks, with clear improvements in both quantitative metrics and qualitative visualizations.

Problem

• Mitigate spurious correlations without auxiliary annotations
• Leverage superclass labels to reduce spurious feature reliance
• Improve domain generalization via disentangled feature representation

Innovation

• Leverages superclass information for feature disentanglement
• Uses gradient-based attention from vision-language model
• Promotes superclass-relevant features for robust prediction