🤖 AI Summary
In multi-source heterogeneous tasks, the coexistence of causal and spurious features leads to out-of-distribution (OOD) generalization failure. Method: We propose a collaborative modeling framework of positive and negative inductive biases: (i) guiding models to learn task-invariant causal features (“what to learn”), and (ii) explicitly suppressing spurious correlations (“what not to learn”). We theoretically prove that existing knowledge fusion methods fail under distributionally robust objectives; accordingly, we design RIME—a novel algorithm integrating causal inference, distributionally robust optimization, meta-learning (a MAML variant), and adversarial disentanglement—to jointly optimize both biases under a family of spurious-feature shifts. Contribution/Results: Experiments demonstrate that RIME significantly improves generalization stability in real-world OOD settings—e.g., cross-hospital medical image prognosis prediction—and achieves state-of-the-art performance in distributionally robust meta-learning.
📝 Abstract
In settings where both spurious and causal predictors are available, standard neural networks trained under the objective of empirical risk minimization (ERM) with no additional inductive biases tend to have a dependence on a spurious feature. As a result, it is necessary to integrate additional inductive biases in order to guide the network toward generalizable hypotheses. Often these spurious features are shared across related tasks, such as estimating disease prognoses from image scans coming from different hospitals, making the challenge of generalization more difficult. In these settings, it is important that methods are able to integrate the proper inductive biases to generalize across both nuisance-varying families as well as task families. Motivated by this setting, we present RIME (Robustly Informed Meta lEarning), a new method for meta learning under the presence of both positive and negative inductive biases (what to learn and what not to learn). We first develop a theoretical causal framework showing why existing approaches at knowledge integration can lead to worse performance on distributionally robust objectives. We then show that RIME is able to simultaneously integrate both biases, reaching state of the art performance under distributionally robust objectives in informed meta-learning settings under nuisance-varying families.