InfoSculpt: Sculpting the Latent Space for Generalized Category Discovery

2026-01-15

AI Summary

This work addresses the challenge in generalized category discovery (GCD) of simultaneously identifying known and novel categories while disentangling semantic features from instance-level noise. To this end, it introduces— for the first time—the information bottleneck principle into GCD in a systematic manner, proposing a dual-scale conditional mutual information (CMI) optimization objective that preserves semantic information at the class level while compressing noise at the instance level. By jointly leveraging labeled and unlabeled data through contrastive learning and data augmentation, the method explicitly decouples class-intrinsic features from instance-specific variations, thereby constructing a more robust and generalizable representation space. Extensive experiments across eight standard benchmarks demonstrate significant performance gains over existing approaches, validating the effectiveness and superiority of the proposed information-theoretic framework.

Technology Category

• Machine Learning: Information Theory
• Computer Vision: Object Detection & Categorization
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Generalized Category Discovery (GCD) aims to classify instances from both known and novel categories within a large-scale unlabeled dataset, a critical yet challenging task for real-world, open-world applications. However, existing methods often rely on pseudo-labeling, or two-stage clustering, which lack a principled mechanism to explicitly disentangle essential, category-defining signals from instance-specific noise. In this paper, we address this fundamental limitation by re-framing GCD from an information-theoretic perspective, grounded in the Information Bottleneck (IB) principle. We introduce InfoSculpt, a novel framework that systematically sculpts the representation space by minimizing a dual Conditional Mutual Information (CMI) objective. InfoSculpt uniquely combines a Category-Level CMI on labeled data to learn compact and discriminative representations for known classes, and a complementary Instance-Level CMI on all data to distill invariant features by compressing augmentation-induced noise. These two objectives work synergistically at different scales to produce a disentangled and robust latent space where categorical information is preserved while noisy, instance-specific details are discarded. Extensive experiments on 8 benchmarks demonstrate that InfoSculpt validating the effectiveness of our information-theoretic approach.

Problem

• Generalized Category Discovery
• latent space disentanglement
• instance-specific noise
• category-defining signals
• unlabeled data

Innovation

• Information Bottleneck
• Conditional Mutual Information
• Latent Space Disentanglement
• Generalized Category Discovery
• Representation Learning