Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation

2025-10-13

AI Summary

To address the challenges of inefficient knowledge transfer and incomplete intermediate-domain data in progressive domain adaptation (GDA), this paper proposes a dynamic weighted self-training framework. The method introduces a time-varying weighting mechanism with an adjustable hyperparameter ρ to adaptively balance the contributions of source-domain supervised loss and target-domain pseudo-label loss, enabling smooth and robust knowledge transfer across intermediate domains. Furthermore, a temporal evolution strategy dynamically schedules domain-specific learning intensity, enhancing model adaptability to progressive distribution shifts. Extensive experiments on Rotated MNIST, Colored MNIST, Portrait, and Cover Type datasets demonstrate substantial improvements over state-of-the-art baselines. Ablation studies confirm that the dynamic scheduling mechanism plays a critical role in mitigating domain shift and improving generalization performance.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this paper, we propose a new method called Self-Training with Dynamic Weighting (STDW), which aims to enhance robustness in Gradual Domain Adaptation (GDA) by addressing the challenge of smooth knowledge migration from the source to the target domain. Traditional GDA methods mitigate domain shift through intermediate domains and self-training but often suffer from inefficient knowledge migration or incomplete intermediate data. Our approach introduces a dynamic weighting mechanism that adaptively balances the loss contributions of the source and target domains during training. Specifically, we design an optimization framework governed by a time-varying hyperparameter $varrho$ (progressing from 0 to 1), which controls the strength of domain-specific learning and ensures stable adaptation. The method leverages self-training to generate pseudo-labels and optimizes a weighted objective function for iterative model updates, maintaining robustness across intermediate domains. Experiments on rotated MNIST, color-shifted MNIST, portrait datasets, and the Cover Type dataset demonstrate that STDW outperforms existing baselines. Ablation studies further validate the critical role of $varrho$'s dynamic scheduling in achieving progressive adaptation, confirming its effectiveness in reducing domain bias and improving generalization. This work provides both theoretical insights and a practical framework for robust gradual domain adaptation, with potential applications in dynamic real-world scenarios. The code is available at https://github.com/Dramwig/STDW.

Problem

• Enhancing robustness in gradual domain adaptation with dynamic weighting
• Addressing inefficient knowledge migration across intermediate domains
• Balancing source and target domain losses during progressive adaptation

Innovation

• Dynamic weighting balances source and target domain losses
• Time-varying hyperparameter controls domain-specific learning strength
• Self-training with pseudo-labels optimizes weighted objective function