FoundObj: Self-supervised Foundation Models as Rewards for Label-free 3D Object Segmentation

2026-05-26

AI Summary

This work proposes FoundObj, a framework for unsupervised multi-category 3D object segmentation in complex point clouds without requiring scene-level manual annotations. FoundObj introduces a novel approach that synergistically integrates semantic and geometric priors from self-supervised 2D/3D foundation models into a reinforcement learning reward mechanism. An object discovery agent operates at the superpoint level, iteratively merging neighboring superpoints under guidance from both semantic consistency and geometric coherence to delineate object instances. Experimental results demonstrate that FoundObj significantly outperforms existing methods across multiple benchmarks and exhibits strong zero-shot and long-tailed generalization capabilities.

Abstract

We address the challenging task of 3D object segmentation in complex scene point clouds without relying on any scene-level human annotations during training. Existing methods are typically constrained to identifying simple objects, primarily due to insufficient object priors in the learning process. In this paper, we present FoundObj, a novel framework featuring a superpoint-based object discovery agent that incrementally merges suitable neighboring superpoints, guided by our innovative semantic and geometric reward modules. These modules synergistically leverage semantic and geometric priors from self-supervised 2D/3D foundation models, providing complementary feedback to the object discovery agent and enabling robust identification of multi-class objects through reinforcement learning. Extensive experiments on diverse benchmarks demonstrate that our approach consistently outperforms existing baselines. Notably, our method exhibits strong generalization in zero-shot and long-tail scenarios, underscoring its potential for scalable, label-free 3D object segmentation.

Problem

• 3D object segmentation
• label-free
• self-supervised
• point clouds
• object discovery

Innovation

• self-supervised foundation models
• label-free 3D segmentation
• superpoint merging
• semantic-geometric reward
• reinforcement learning