DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning

2026-04-02

AI Summary

This work addresses the disconnection between feature learning and clustering assignment in deep clustering by proposing the first end-to-end differentiable unsupervised prototype representation learning framework. The approach fully internalizes prototype generation as a network component, eliminating the need for external clustering algorithms or discrete pseudo-labels. It introduces a dual-competition layer to enable soft clustering assignments and jointly optimizes a composite objective comprising soft quantization loss decomposition, simplex-constrained reconstruction error, and a prototype variance term. This formulation reveals an intrinsic self-regulating mechanism within the loss function that effectively prevents prototype collapse. Theoretical analysis confirms the mechanism’s validity across millions of training iterations, exhibiting a strong negative feedback correlation of −0.98. Empirically, the method achieves a 65% improvement in clustering accuracy over non-differentiable ablated models and a 122% gain compared to DeepCluster.

Abstract

A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment. Most architectures invoke an external clustering step, typically k-means, to produce pseudo-labels that guide training, preventing the backbone from directly optimising for cluster quality. This paper introduces Deep Dual Competitive Learning (DDCL), the first fully differentiable end-to-end framework for unsupervised prototype-based representation learning. The core contribution is architectural: the external k-means is replaced by an internal Dual Competitive Layer (DCL) that generates prototypes as native differentiable outputs of the network. This single inversion makes the complete pipeline, from backbone feature extraction through prototype generation to soft cluster assignment, trainable by backpropagation through a single unified loss, with no Lloyd iterations, no pseudo-label discretisation, and no external clustering step. To ground the framework theoretically, the paper derives an exact algebraic decomposition of the soft quantisation loss into a simplex-constrained reconstruction error and a non-negative weighted prototype variance term. This identity reveals a self-regulating mechanism built into the loss geometry: the gradient of the variance term acts as an implicit separation force that resists prototype collapse without any auxiliary objective, and leads to a global Lyapunov stability theorem for the reduced frozen-encoder system. Six blocks of controlled experiments validate each structural prediction. The decomposition identity holds with zero violations across more than one hundred thousand training epochs; the negative feedback cycle is confirmed with Pearson -0.98; with a jointly trained backbone, DDCL outperforms its non-differentiable ablation by 65% in clustering accuracy and DeepCluster end-to-end by 122%.

Problem

• deep clustering
• feature learning
• cluster assignment
• unsupervised representation learning
• prototype collapse

Innovation

• differentiable clustering
• prototype-based learning
• end-to-end unsupervised learning
• dual competitive layer
• soft quantisation loss