Are We Done with Object-Centric Learning?

2025-04-09

AI Summary

This work addresses the poorly understood relationship between object separation capability and high-level objectives—particularly out-of-distribution (OOD) generalization—in object-centric learning (OCL), focusing on spurious background cues as a key bottleneck for OOD generalization. We propose OCCAM (Mask-Driven Object-Centric Classification Analysis), a novel, training-free probing method that systematically evaluates object separation efficacy from an OOD generalization perspective for the first time. Experiments demonstrate that pixel-level object masks generated by modern segmentation models, when used as object representations, substantially outperform conventional slot-based OCL approaches on OOD object discovery benchmarks, achieving superior zero-shot performance. These results empirically confirm that segmentation-driven object representations have effectively realized the core objective of OCL. To support cognitive modeling and applied research, we open-source a modular, extensible toolkit implementing OCCAM and related analyses.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Representation Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Object-centric learning (OCL) seeks to learn representations that only encode an object, isolated from other objects or background cues in a scene. This approach underpins various aims, including out-of-distribution (OOD) generalization, sample-efficient composition, and modeling of structured environments. Most research has focused on developing unsupervised mechanisms that separate objects into discrete slots in the representation space, evaluated using unsupervised object discovery. However, with recent sample-efficient segmentation models, we can separate objects in the pixel space and encode them independently. This achieves remarkable zero-shot performance on OOD object discovery benchmarks, is scalable to foundation models, and can handle a variable number of slots out-of-the-box. Hence, the goal of OCL methods to obtain object-centric representations has been largely achieved. Despite this progress, a key question remains: How does the ability to separate objects within a scene contribute to broader OCL objectives, such as OOD generalization? We address this by investigating the OOD generalization challenge caused by spurious background cues through the lens of OCL. We propose a novel, training-free probe called $ extbf{Object-Centric Classification with Applied Masks (OCCAM)}$, demonstrating that segmentation-based encoding of individual objects significantly outperforms slot-based OCL methods. However, challenges in real-world applications remain. We provide the toolbox for the OCL community to use scalable object-centric representations, and focus on practical applications and fundamental questions, such as understanding object perception in human cognition. Our code is available $href{https://github.com/AlexanderRubinstein/OCCAM}{here}$.

Problem

• Investigates OOD generalization in object-centric learning
• Proposes OCCAM for segmentation-based object encoding
• Addresses challenges in real-world OCL applications

Innovation

• Segmentation-based encoding for object separation
• Training-free OCCAM probe for classification
• Scalable object-centric representation toolbox