CObL: Toward Zero-Shot Ordinal Layering without User Prompting

2025-08-11

AI Summary

This work addresses unsupervised hierarchical scene parsing from a single natural image: inferring a multi-object layer representation ordered by occlusion, without prior knowledge of object count or user guidance, and generalizing zero-shot to real-world scenes with arbitrary object numbers and categories. To this end, we propose CObL, a Stable Diffusion–based diffusion model that introduces inference-time guidance to jointly leverage synthetic supervision and natural-image priors for end-to-end pixel-level layered generation. Our method is the first to support parallel, variable-depth layer decomposition—overcoming traditional constraints of fixed object counts and domain-specific training. Evaluated on real desktop scenes, CObL achieves high-fidelity unsupervised separation, accurate occluded-region inpainting, and robust depth-order recovery, with low reconstruction error and strong cross-scene generalization.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Vision benefits from grouping pixels into objects and understanding their spatial relationships, both laterally and in depth. We capture this with a scene representation comprising an occlusion-ordered stack of "object layers," each containing an isolated and amodally-completed object. To infer this representation from an image, we introduce a diffusion-based architecture named Concurrent Object Layers (CObL). CObL generates a stack of object layers in parallel, using Stable Diffusion as a prior for natural objects and inference-time guidance to ensure the inferred layers composite back to the input image. We train CObL using a few thousand synthetically-generated images of multi-object tabletop scenes, and we find that it zero-shot generalizes to photographs of real-world tabletops with varying numbers of novel objects. In contrast to recent models for amodal object completion, CObL reconstructs multiple occluded objects without user prompting and without knowing the number of objects beforehand. Unlike previous models for unsupervised object-centric representation learning, CObL is not limited to the world it was trained in.

Problem

• Inferring occlusion-ordered object layers from images
• Achieving zero-shot generalization to real-world tabletop scenes
• Reconstructing multiple occluded objects without user prompting

Innovation

• Diffusion-based architecture for object layers
• Stable Diffusion prior for natural objects
• Inference-time guidance for image compositing