VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAM

📅 2026-09-01
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
VOIM通过积累未修改的感知证据来构建开放词汇3D实例地图,解决了无需训练即可从RGB-D或单目RGB生成3D实例映射的问题。
📝 Abstract
We present Voxel-Grounded Online Instance Manager (VOIM), a training-free voxel-grounded instance manager that builds open-vocabulary 3D instance maps from RGB-D or from monocular RGB alone, a regime no prior training-free system addresses. Online systems typically segment object instances and label them at first detection, committing when evidence is weakest. VOIM instead defers label and instance decisions until soft evidence from unmodified, off-the-shelf perception has accumulated per voxel across views. We show that the mapping stage, rather than the particular perception models, carries the result: across four perception configurations on ScanNet++, varying the region descriptor, the detector label prior and the mask source, the map exceeds the strongest online RGB-D system, OVO-SLAM, by between 4.8 and 11.7 mIoU. Perception is not neutral, and substituting that baseline's own descriptor family costs 4.1 of the margin, yet the baseline carries the marginally better 2D descriptor (33.7 vs. 31.5 mIoU over three scenes) and still realizes the weaker map. Under a like-for-like protocol VOIM reaches 44.07 mIoU on ScanNet++ against 32.37, winning all ten scenes and both aggregations (pooled 33.31 vs. 25.97), and the same system runs unchanged to fully monocular RGB, matching that baseline pooled on Replica (27.80 vs. 27.50). The advantage is regime-specific: under Replica's all-classes scoring, matched inputs give a split result, 28.60 vs. 27.50 pooled against 24.59 vs. 30.11 on the per-scene mean. Room scale is label-limited and building scale drift-limited. Labeling does not run in real time, dominated by per-class detection over the full vocabulary. The maps export occupancy grids and resolve free-form queries to object instances.
Problem

Research questions and friction points this paper is trying to address.

open-vocabulary
3D instance mapping
training-free
RGB-D
monocular
Innovation

Methods, ideas, or system contributions that make the work stand out.

Training-Free
Open-Vocabulary 3D Instance Mapping
Voxel-Grounded
RGB-D and Monocular SLAM
Online Instance Manager
🔎 Similar Papers
💼 Related Jobs
No related jobs found.
S
Sangmin Song
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo NSW 2007, Australia
S
Sarath Kodagoda
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo NSW 2007, Australia
M
Marc G. Carmichael
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo NSW 2007, Australia
K
Karthick Thiyagarajan
Smart Sensing and Robotics Laboratory (SensR Lab), Centre for Advanced Manufacturing Technology, School of Engineering, Design and Built Environment, Western Sydney University, Kingswood NSW 2747, Australia
A
Amal Gunatilake
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo NSW 2007, Australia
K
Kelly Prentice
Guide Dogs New South Wales/Australian Capital Territory, Sydney NSW, Australia
J
Jodi Martin
Guide Dogs New South Wales/Australian Capital Territory, Sydney NSW, Australia