Exploring a Principled Framework for Deep Subspace Clustering

2025-03-21

AI Summary

Existing deep subspace clustering (DSC) methods rely on the strict union-of-subspaces (UoS) assumption and suffer from feature collapse during joint representation and self-expression coefficient learning, lacking theoretical guarantees. This paper proposes PRO-DSC—the first end-to-end DSC framework that simultaneously provides rigorous theoretical guarantees and structured modeling. Its core contribution is the first theoretical proof that regularized self-expression models prevent feature collapse and ensure that optimal representations exactly lie in an orthogonal UoS. PRO-DSC integrates structured representation regularization, differentiable self-expression modeling, and an efficient optimization algorithm. Extensive experiments on multiple benchmark datasets demonstrate significant improvements over state-of-the-art methods. Empirical results consistently validate the theoretical claims, confirming both the efficacy and soundness of the proposed framework. The implementation is publicly available.

Technology Category

• Machine Learning: Learning with Manifolds
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Computer Vision: Representation Learning for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Subspace clustering is a classical unsupervised learning task, built on a basic assumption that high-dimensional data can be approximated by a union of subspaces (UoS). Nevertheless, the real-world data are often deviating from the UoS assumption. To address this challenge, state-of-the-art deep subspace clustering algorithms attempt to jointly learn UoS representations and self-expressive coefficients. However, the general framework of the existing algorithms suffers from a catastrophic feature collapse and lacks a theoretical guarantee to learn desired UoS representation. In this paper, we present a Principled fRamewOrk for Deep Subspace Clustering (PRO-DSC), which is designed to learn structured representations and self-expressive coefficients in a unified manner. Specifically, in PRO-DSC, we incorporate an effective regularization on the learned representations into the self-expressive model, prove that the regularized self-expressive model is able to prevent feature space collapse, and demonstrate that the learned optimal representations under certain condition lie on a union of orthogonal subspaces. Moreover, we provide a scalable and efficient approach to implement our PRO-DSC and conduct extensive experiments to verify our theoretical findings and demonstrate the superior performance of our proposed deep subspace clustering approach. The code is available at https://github.com/mengxianghan123/PRO-DSC.

Problem

• Addressing deviation from UoS assumption in real-world data
• Preventing feature collapse in deep subspace clustering
• Learning structured representations with theoretical guarantees

Innovation

• Unified framework for deep subspace clustering
• Regularization prevents feature space collapse
• Learned representations lie on orthogonal subspaces