From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matching

2025-05-30

AI Summary

Spurious correlations degrade model generalization across environments; existing invariant learning methods rely on test-time data and inadequately enforce invariance at the representation level. This paper proposes an “invariant data pair”-driven data augmentation paradigm, pioneering the shift of invariance constraints from representation to data level: it generates noise-augmented counterfactual sample pairs via linear causal modeling and counterfactual reasoning, ensuring consistency under robust classifiers. We introduce the Noise Counterfactual Matching (NCM) constraint and theoretically prove that a single invariant pair eliminates one spurious feature; generalization error is bounded by controllable upper limits dependent on counterfactual diversity and quality. The method requires no test-time data or architectural modifications—only linear probe fine-tuning. It significantly outperforms IRM and other baselines on both synthetic and real-world benchmarks, maintaining stable robust gains even with small-scale invariant pairs.

Technology Category

• Machine Learning: Causal Learning
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Spurious correlations can cause model performance to degrade in new environments. Prior causality-inspired works aim to learn invariant representations (e.g., IRM) but typically underperform empirical risk minimization (ERM). Recent alternatives improve robustness by leveraging test-time data, but such data may be unavailable in practice. To address these issues, we take a data-centric approach by leveraging invariant data pairs, pairs of samples that would have the same prediction with the optimally robust classifier. We prove that certain counterfactual pairs will naturally satisfy this invariance property and introduce noisy counterfactual matching (NCM), a simple constraint-based method for leveraging invariant pairs for enhanced robustness, even with a small set of noisy pairs-in the ideal case, each pair can eliminate one spurious feature. For linear causal models, we prove that the test domain error can be upper bounded by the in-domain error and a term that depends on the counterfactuals' diversity and quality. We validate on a synthetic dataset and demonstrate on real-world benchmarks that linear probing on a pretrained backbone improves robustness.

Problem

• Addressing model degradation from spurious correlations across environments
• Improving robustness without relying on test-time data availability
• Leveraging invariant data pairs to eliminate spurious features

Innovation

• Leveraging invariant data pairs for robustness
• Noisy counterfactual matching enhances model robustness
• Linear probing improves robustness on pretrained models