Domain Generalizable Continual Learning

2025-10-19

AI Summary

Existing continual learning (CL) methods assume consistent training and testing domains, leading to significant performance degradation in domain-generalized continual learning (DGCL)—a challenging setting where models learn sequential single-domain tasks yet must generalize to multiple unseen domains. This paper introduces a novel framework for domain-generalizable continual learning, centered on DoT (Domain-adaptive Transformation), an adaptive domain transformation method. DoT decouples semantic and domain-specific feature representations and achieves cross-domain output alignment via a brain-inspired distributed-plus-hub mechanism. Built upon pretrained foundation models, DoT supports both full-parameter and parameter-efficient fine-tuning and offers plug-and-play integration. Experiments across multiple DGCL benchmarks demonstrate that DoT substantially enhances the cross-domain generalization capability of mainstream CL methods, effectively accumulates transferable knowledge, and achieves strong efficiency–lightweightness trade-offs.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

To adapt effectively to dynamic real-world environments, intelligent systems must continually acquire new skills while generalizing them to diverse, unseen scenarios. Here, we introduce a novel and realistic setting named domain generalizable continual learning (DGCL): a model learns sequential tasks with each involving a single domain, aiming to perform well across all encountered tasks and domains. This setting poses unique challenges in acquiring, retaining, and leveraging both semantic- and domain-relevant information for robust generalization. Although state-of-the-art continual learning (CL) methods have employed pre-trained models (PTMs) to enhance task-specific generalization, they typically assume identical training and testing domains for each task and therefore perform poorly in DGCL. To this end, we propose adaptive Domain Transformation (DoT), an innovative PTMs-based approach tailored to DGCL. Inspired by the distributed-plus-hub theory of the human brain, DoT disentangles semantic- and domain-relevant information in representation learning, and adaptively transforms task representations across various domains for output alignment, ensuring balanced and generalized predictions. DoT serves as a plug-in strategy that greatly facilitates state-of-the-art CL baselines under both full parameter tuning and parameter-efficient tuning paradigms in DGCL, validated by extensive experiments. Also, DoT is shown to accumulate domain-generalizable knowledge from DGCL, and ensure resource efficiency with a lightweight implementation.

Problem

• Continual learning across sequential tasks with single domains
• Generalizing learned skills to diverse unseen scenarios
• Disentangling semantic and domain information for robust predictions

Innovation

• Adaptive Domain Transformation disentangles semantic and domain information
• Plug-in strategy enhances continual learning baselines across domains
• Lightweight implementation ensures resource efficiency and generalizable knowledge