Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

2026-06-05

AI Summary

This work addresses catastrophic forgetting in unsupervised continual learning, where the absence of labels and inaccessibility of historical data pose significant challenges. To this end, it formally introduces the task of Unsupervised Continual Clustering (UCC) for the first time. The authors propose a buffer-free forward–backward knowledge distillation mechanism that leverages a continually evolving teacher network and lightweight task-specific student networks, augmented with clustering projection heads. This framework effectively preserves previously learned cluster structures while adapting to new ones. Evaluated on four benchmark datasets, the method substantially outperforms existing continual learning baselines, achieving higher clustering accuracy across sequential tasks and significantly mitigating forgetting.

Abstract

Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks upon learning new ones. This challenge is amplified in UCL due to the absence of labels to guide learning and memory retention. Existing mitigation strategies, such as knowledge distillation and replay buffers, often raise memory and privacy concerns. Moreover, current UCL methods largely overlook clustering-specific objectives. To fill this gap, we introduce Unsupervised Continual Clustering (UCC) and propose Forward-Backward Knowledge Distillation for Continual Clustering (FBCC). FBCC employs a continual teacher network with a clustering projector and lightweight task-specific students. Through a dual-phase forward-backward distillation process, the teacher learns new clusters while preserving previously discovered cluster structure without storing past data. FBCC represents a pioneering approach to UCC, demonstrating improved clustering performance across sequential tasks. Experiments on four benchmark datasets demonstrate that FBCC consistently outperforms existing continual learning baselines in clustering accuracy while significantly reducing catastrophic forgetting.

Problem

• Unsupervised Continual Learning
• Catastrophic Forgetting
• Continual Clustering
• Knowledge Distillation
• Cluster Preservation

Innovation

• Unsupervised Continual Clustering
• Forward-Backward Knowledge Distillation
• Catastrophic Forgetting
• Continual Learning
• Clustering