Interacted Planes Reveal 3D Line Mapping

2026-02-01

AI Summary

This work proposes LiP-Map, a novel framework for reconstructing structured 3D line segment maps from multi-view RGB images. LiP-Map is the first method to explicitly incorporate planar topological structure into 3D line reconstruction by jointly optimizing learnable line and plane primitives, thereby modeling their geometric and topological interactions without relying on pairwise coplanarity constraints. By integrating multi-view geometric constraints with topological reasoning, LiP-Map significantly outperforms existing approaches across five benchmarks—including ScanNetV2 and ScanNet++—achieving consistent improvements in both accuracy and completeness. Furthermore, the reconstructed line segments effectively enhance visual localization performance when used as geometric priors.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services

Abstract

3D line mapping from multi-view RGB images provides a compact and structured visual representation of scenes. We study the problem from a physical and topological perspective: a 3D line most naturally emerges as the edge of a finite 3D planar patch. We present LiP-Map, a line-plane joint optimization framework that explicitly models learnable line and planar primitives. This coupling enables accurate and detailed 3D line mapping while maintaining strong efficiency (typically completing a reconstruction in 3 to 5 minutes per scene). LiP-Map pioneers the integration of planar topology into 3D line mapping, not by imposing pairwise coplanarity constraints but by explicitly constructing interactions between plane and line primitives, thus offering a principled route toward structured reconstruction in man-made environments. On more than 100 scenes from ScanNetV2, ScanNet++, Hypersim, 7Scenes, and Tanks\&Temple, LiP-Map improves both accuracy and completeness over state-of-the-art methods. Beyond line mapping quality, LiP-Map significantly advances line-assisted visual localization, establishing strong performance on 7Scenes. Our code is released at https://github.com/calmke/LiPMAP for reproducible research.

Problem

• 3D line mapping
• multi-view RGB images
• planar topology
• structured reconstruction
• visual localization

Innovation

• 3D line mapping
• planar primitives
• line-plane interaction
• structured reconstruction
• joint optimization