Single Domain Generalization with Model-aware Parametric Batch-wise Mixup

2025-02-22

AI Summary

Single-domain generalization (SDG) faces the core challenge of severe distribution shift between the source domain and unseen target domains, leading to poor generalization. To address this, we propose Model-aware Parameterized Batch-wise Mixing (MPBM), a novel data augmentation framework that integrates adversarial query guidance, attention-driven generative networks, and Stochastic Gradient Langevin Dynamics (SGLD) into a controllable synthesis mechanism—enabling generation of semantically consistent and distribution-aligned augmented samples. MPBM extends standard Mixup via parameterized interpolation, broadening the representational coverage of original data and mitigating inter-domain mismatch. Evaluated on multiple standard SDG benchmarks, MPBM consistently outperforms existing methods, achieving state-of-the-art performance. It significantly enhances model robustness and generalization capability on unseen target domains, demonstrating strong practical efficacy for real-world deployment.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Single Domain Generalization (SDG) remains a formidable challenge in the field of machine learning, particularly when models are deployed in environments that differ significantly from their training domains. In this paper, we propose a novel data augmentation approach, named as Model-aware Parametric Batch-wise Mixup (MPBM), to tackle the challenge of SDG. MPBM deploys adversarial queries generated with stochastic gradient Langevin dynamics, and produces model-aware augmenting instances with a parametric batch-wise mixup generator network that is carefully designed through an innovative attention mechanism. By exploiting inter-feature correlations, the parameterized mixup generator introduces additional versatility in combining features across a batch of instances, thereby enhancing the capacity to generate highly adaptive and informative synthetic instances for specific queries. The synthetic data produced by this adaptable generator network, guided by informative queries, is expected to significantly enrich the representation space covered by the original training dataset and subsequently enhance the prediction model's generalizability across diverse and previously unseen domains. To prevent excessive deviation from the training data, we further incorporate a real-data alignment-based adversarial loss into the learning process of MPBM, regularizing any tendencies toward undesirable expansions. We conduct extensive experiments on several benchmark datasets. The empirical results demonstrate that by augmenting the training set with informative synthesis data, our proposed MPBM method achieves the state-of-the-art performance for single domain generalization.

Problem

• Single Domain Generalization challenge
• Model-aware Parametric Batch-wise Mixup
• Enhancing model generalizability across domains

Innovation

• Parametric Batch-wise Mixup
• Adversarial query generation
• Real-data alignment loss