Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer

2025-02-27

AI Summary

In open-world continual learning (OWCL), jointly modeling known-class classification and unknown-sample detection remains challenging due to incomplete training data and underutilization of unknown knowledge. This paper first uncovers a strong intrinsic correlation between known and unknown representations in the embedding space. To address these issues, we propose Holistic Knowns-Unknowns Knowledge Transfer (HoliTrans), a unified framework that enhances feature linear separability via nonlinear random projection (NRP) and constructs a dynamic, distribution-aware prototype space (DAPs) for adaptive knowledge transfer across both known and unknown samples. Evaluated on multiple OWCL benchmarks, HoliTrans consistently outperforms 22 state-of-the-art baselines. It effectively bridges the gap between theoretical design and practical deployment, significantly improving model robustness, generalization, and scalability under realistic open-world constraints.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Open-World Continual Learning (OWCL) is a challenging paradigm where models must incrementally learn new knowledge without forgetting while operating under an open-world assumption. This requires handling incomplete training data and recognizing unknown samples during inference. However, existing OWCL methods often treat open detection and continual learning as separate tasks, limiting their ability to integrate open-set detection and incremental classification in OWCL. Moreover, current approaches primarily focus on transferring knowledge from known samples, neglecting the insights derived from unknown/open samples. To address these limitations, we formalize four distinct OWCL scenarios and conduct comprehensive empirical experiments to explore potential challenges in OWCL. Our findings reveal a significant interplay between the open detection of unknowns and incremental classification of knowns, challenging a widely held assumption that unknown detection and known classification are orthogonal processes. Building on our insights, we propose extbf{HoliTrans} (Holistic Knowns-Unknowns Knowledge Transfer), a novel OWCL framework that integrates nonlinear random projection (NRP) to create a more linearly separable embedding space and distribution-aware prototypes (DAPs) to construct an adaptive knowledge space. Particularly, our HoliTrans effectively supports knowledge transfer for both known and unknown samples while dynamically updating representations of open samples during OWCL. Extensive experiments across various OWCL scenarios demonstrate that HoliTrans outperforms 22 competitive baselines, bridging the gap between OWCL theory and practice and providing a robust, scalable framework for advancing open-world learning paradigms.

Problem

• Integrate open-set detection and incremental classification.
• Address knowledge transfer from unknown/open samples.
• Develop a scalable framework for Open-World Continual Learning.

Innovation

• Integrates nonlinear random projection
• Uses distribution-aware prototypes
• Dynamic updates open sample representations