Emergent 3D Instance Segmentation from Self-Supervised Point Transformers

2026-08-16

AI Summary

This study addresses the reliance on handcrafted geometric priors in unsupervised 3D instance segmentation for outdoor LiDAR by proposing TokenGraph3D. Leveraging the observation that attention keys in self-supervised point Transformers encode instance signals under Rotary Position Embedding (RoPE), this method constructs a key similarity graph and applies connected component analysis to achieve training-free graph grouping without density clustering priors. Experiments demonstrate that TokenGraph3D significantly outperforms feature-based baselines in prior-free settings, successfully enabling the emergence of 3D instance structures across multiple datasets. Consequently, this approach effectively overcomes the limitations of traditional methods dependent on manually designed rules, offering a robust solution for unsupervised outdoor scene understanding.

Abstract

Unsupervised 3D instance segmentation of outdoor LiDAR scans has traditionally relied on handcrafted geometric priors such as density-based clustering, motion cues, or projected 2D detections. In this work, we investigate whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior. Using this transformer purely as a feature extractor, we probe its internal representations across the SemanticKITTI, nuScenes, and Waymo Perception datasets. Our analysis yields four core insights: (1) the instance signal concentrates in the attention queries and keys rather than in the values or final output features; (2) output features semantically collapse, merging adjacent same-class objects that the queries and keys keep distinct; (3) this instance signal is bimodal in depth, strongest at the shallowest and deepest encoder stages; and (4) this signal is driven predominantly by the rotary position encoding (RoPE), whose removal collapses its advantage. We put these findings into our method TokenGraph3D, a training-free segmenter that groups points via connected components on a key-similarity graph, using neither density-based clustering nor proximity priors. Under identical prior-free conditions, we substantially outperform output-feature baselines, making the emergent 3D instance structure visible.

Problem

• Unsupervised 3D Instance Segmentation
• Self-Supervised Point Transformer
• Outdoor LiDAR Scans
• Handcrafted Geometric Priors

Innovation

• Self-Supervised Point Transformer
• Training-Free Segmentation
• Attention Keys
• Rotary Position Encoding
• TokenGraph3D