Lookalike3D: Seeing Double in 3D

2026-03-25

AI Summary

This work addresses the overlooked challenge of identifying repeated or near-identical objects in indoor 3D scenes—a visual cue that can significantly enhance scene understanding consistency. We propose Lookalike3D, the first method to define and tackle the lookalike object detection task in 3D indoor environments, which classifies object pairs into “same,” “similar,” or “different” categories using a multi-view image Transformer augmented with semantic priors from large-scale foundation models. To support this task, we introduce 3DTwins, a novel dataset comprising 76,000 meticulously annotated object pairs. Furthermore, we demonstrate how lookalike cues can be effectively integrated into downstream applications such as 3D reconstruction and part co-segmentation. Experiments show that our approach improves IoU by 104% over baseline methods on 3DTwins, substantially enhancing the consistency and quality of 3D scene understanding.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

3D object understanding and generation methods produce impressive results, yet they often overlook a pervasive source of information in real-world scenes: repeated objects. We introduce the task of lookalike object detection in indoor scenes, which leverages repeated and complementary cues from identical and near-identical object pairs. Given an input scene, the task is to classify pairs of objects as identical, similar or different using multiview images as input. To address this, we present Lookalike3D, a multiview image transformer that effectively distinguishes such object pairs by harnessing strong semantic priors from large image foundation models. To support this task, we collected the 3DTwins dataset, containing 76k manually annotated identical, similar and different pairs of objects based on ScanNet++, and show an improvement of 104% IoU over baselines. We demonstrate how our method improves downstream tasks such as enabling joint 3D object reconstruction and part co-segmentation, turning repeated and lookalike objects into a powerful cue for consistent, high-quality 3D perception. Our code, dataset and models will be made publicly available.

Problem

• lookalike object detection
• 3D object understanding
• repeated objects
• indoor scenes
• object similarity

Innovation

• lookalike object detection
• multiview image transformer
• 3D object understanding
• semantic priors
• 3DTwins dataset