Identifying Latent Concepts and Structures for Generalized Category Discovery

2026-07-01

AI Summary

This work addresses the limitations of existing generalized category discovery (GCD) methods, which rely on standard vision backbones that produce high-rank, entangled features obscuring latent semantic structure and hindering novel class discovery. To overcome this, we propose Compositional Primitive Fields (CPF-GCD), a framework that introduces a spatial field mechanism between the backbone and prediction head to disentangle images into reusable visual primitives and their spatial layouts. By explicitly modeling shared semantic structure through low-rank compositions, CPF represents both known and novel categories as activation patterns over a common primitive vocabulary. CPF is the first to leverage low-rank compositionality as a key inductive bias for open-world recognition, shifting the paradigm from holistic embeddings to atomic part-based composition. As a plug-and-play module, CPF consistently achieves significant performance gains across multiple GCD benchmarks, demonstrating the efficacy of structured compositional representations for discovering new categories.

Abstract

Generalized Category Discovery (GCD) aims to recognize known classes while autonomously discovering novel ones in open-world settings. However, current approaches primarily focus on designing clustering objectives, often overlooking a critical bottleneck: standard vision backbones yield high-rank, entangled token representations that are ill-suited for unsupervised discovery of latent concepts and structures. In this paper, we propose Compositional Primitive Fields (CPF-GCD), a novel representation learning framework that reshapes the feature space to make such latent structure identifiable by enforcing a low-rank compositional organization. Our core hypothesis is that all categories, whether known or novel, can be expressed as compositions and spatial arrangements of a finite set of learnable visual primitives that capture reusable concepts. CPF instantiates this geometric constraint via a spatial field mechanism. Inserted between the backbone and the head, it rewrites noisy patch tokens through low-rank primitive mixtures, effectively decomposing images into reusable atomic parts and their spatial layouts. By explicitly modeling the spatial distribution of primitives, CPF enables novel categories to emerge naturally as new activation patterns over a shared vocabulary. This shifts the focus of representation from merely partitioning global embeddings to constructing a structured and separable primitive field. Extensive experiments demonstrate that CPF serves as a generic, plug-and-play module that consistently boosts performance across diverse GCD baselines, validating that identifying and leveraging low-rank compositional structure is a crucial inductive bias for open-world recognition.

Problem

• Generalized Category Discovery
• latent concepts
• representation learning
• unsupervised discovery
• compositional structure

Innovation

• Compositional Primitive Fields
• Generalized Category Discovery
• low-rank representation
• visual primitives
• spatial field mechanism