Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyond

2026-04-24

AI Summary

This work addresses the limitations of existing panoramic 3D reconstruction methods, which struggle with spherical distortion and a lack of real-world data featuring continuous camera trajectories, thereby hindering feed-forward multi-view modeling. To overcome these challenges, we present the first large-scale, continuously captured panoramic 3D dataset, comprising 109,495 360° images accompanied by accurately registered point clouds, meshes, camera poses, and—uniquely—high-completeness depth maps. Our data acquisition pipeline integrates LiDAR scanning and 360° cameras, leveraging both online and offline SLAM for robust pose estimation. We further introduce a tailored post-processing pipeline for panoramic data, including geometric denoising, hole filling, and remeshing, which substantially enhances reconstruction fidelity. This dataset and its associated benchmark significantly advance training and evaluation of panoramic 3D reconstruction models.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

While feed-forward 3D reconstruction models have advanced rapidly, they still exhibit degraded performance on panoramas due to spherical distortions. Moreover, existing panoramic 3D datasets are predominantly collected with 360 cameras fixed at discrete locations, resulting in discontinuous trajectories. These limitations critically hinder the development of panoramic feed-forward 3D reconstruction, especially for the multi-view setting. In this paper, we present Holo360D, a comprehensive dataset containing 109,495 panoramas paired with registered point clouds, meshes, and aligned camera poses. To our knowledge, Holo360D is the first large-scale dataset that provides continuous panoramic sequences with accurately aligned high-completeness depth maps. The raw data are initially collected using a 3D laser scanner coupled with a 360 camera. Subsequently, the raw data are processed with both online and offline SLAM systems. Furthermore, to enhance the 3D data quality, a post-processing pipeline tailored for the 360 dataset is proposed, including geometry denoising, mesh hole filling, and region-specific remeshing. Finally, we establish a new benchmark by fine-tuning 3D reconstruction models on Holo360D, providing key insights into effective fine-tuning strategies. Our results demonstrate that Holo360D delivers superior training signals and provides a comprehensive benchmark for advancing panoramic 3D reconstruction models. Datasets and Code will be made publicly available.

Problem

• panoramic 3D reconstruction
• spherical distortions
• discontinuous trajectories
• 360 datasets
• multi-view setting

Innovation

• panoramic 3D reconstruction
• continuous trajectory
• large-scale dataset
• 360-degree SLAM
• geometry post-processing