F2M-Reg: Unsupervised RGB-D Point Cloud Registration with Frame-to-Model Optimization

2024-05-01

AI Summary

This paper addresses unsupervised RGB-D point cloud registration by proposing a novel frame-to-model (F2M) optimization framework, designed to overcome limitations of conventional frame-to-frame methods—particularly their sensitivity to inter-frame inconsistencies such as illumination changes, occlusions, and specular reflections. The method constructs a global scene representation using a neural implicit field (NeRF-style), jointly optimizing the pose of each input frame into this unified model. It introduces the first unsupervised F2M training paradigm, eliminating error propagation caused by frame-to-frame coupling. Additionally, it employs a Sim-RGBD synthetic data warm-start strategy to improve generalization to real-world scenes. Evaluated on four standard benchmarks, the approach significantly outperforms state-of-the-art methods, especially under challenging conditions including severe illumination variations and low geometric overlap. The source code and pretrained models are publicly available.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This work studies the problem of unsupervised RGB-D point cloud registration, which aims at training a robust registration model without ground-truth pose supervision. Existing methods usually leverages unposed RGB-D sequences and adopt a frame-to-frame framework based on differentiable rendering to train the registration model, which enforces the photometric and geometric consistency between the two frames for supervision. However, this frame-to-frame framework is vulnerable to inconsistent factors between different frames, e.g., lighting changes, geometry occlusion, and reflective materials, which leads to suboptimal convergence of the registration model. In this paper, we propose a novel frame-to-model optimization framework named F2M-Reg for unsupervised RGB-D point cloud registration. We leverage the neural implicit field as a global model of the scene and optimize the estimated poses of the frames by registering them to the global model, and the registration model is subsequently trained with the optimized poses. Thanks to the global encoding capability of neural implicit field, our frame-to-model framework is significantly more robust to inconsistent factors between different frames and thus can provide better supervision for the registration model. Besides, we demonstrate that F2M-Reg can be further enhanced by a simplistic synthetic warming-up strategy. To this end, we construct a photorealistic synthetic dataset named Sim-RGBD to initialize the registration model for the frame-to-model optimization on real-world RGB-D sequences. Extensive experiments on four challenging benchmarks have shown that our method surpasses the previous state-of-the-art counterparts by a large margin, especially under scenarios with severe lighting changes and low overlap. Our code and models are available at https://github.com/MrIsland/F2M_Reg.

Problem

• Unsupervised RGB-D point cloud registration without pose supervision
• Robust registration model training using frame-to-model optimization
• Addressing inconsistent factors like lighting changes and occlusion

Innovation

• Unsupervised RGB-D point cloud registration
• Frame-to-model optimization with neural implicit field
• Synthetic warming-up strategy using Sim-RGBD dataset