OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Control

2025-11-04

AI Summary

Existing inverse rendering and relighting methods heavily rely on synthetic data, limiting realism and generalization to real-world scenes. To address this, OLATverse introduces the first large-scale, high-fidelity real-object dataset, comprising 765 material categories and approximately 9 million multi-view images captured under precisely controlled illumination using 35 cameras and 331 programmable light sources. It establishes a novel, scalable acquisition paradigm for real objects under high-precision lighting control. The dataset provides comprehensive annotations—including camera calibration parameters, object masks, surface normals, and diffuse albedo—along with a unified benchmark. As the largest publicly available real-object dataset to date, OLATverse significantly improves the performance and generalization of inverse rendering and normal estimation models on real-world imagery. It advances vision research grounded in authentic physical data, enabling more robust and realistic scene understanding.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce OLATverse, a large-scale dataset comprising around 9M images of 765 real-world objects, captured from multiple viewpoints under a diverse set of precisely controlled lighting conditions. While recent advances in object-centric inverse rendering, novel view synthesis and relighting have shown promising results, most techniques still heavily rely on the synthetic datasets for training and small-scale real-world datasets for benchmarking, which limits their realism and generalization. To address this gap, OLATverse offers two key advantages over existing datasets: large-scale coverage of real objects and high-fidelity appearance under precisely controlled illuminations. Specifically, OLATverse contains 765 common and uncommon real-world objects, spanning a wide range of material categories. Each object is captured using 35 DSLR cameras and 331 individually controlled light sources, enabling the simulation of diverse illumination conditions. In addition, for each object, we provide well-calibrated camera parameters, accurate object masks, photometric surface normals, and diffuse albedo as auxiliary resources. We also construct an extensive evaluation set, establishing the first comprehensive real-world object-centric benchmark for inverse rendering and normal estimation. We believe that OLATverse represents a pivotal step toward integrating the next generation of inverse rendering and relighting methods with real-world data. The full dataset, along with all post-processing workflows, will be publicly released at https://vcai.mpi-inf.mpg.de/projects/OLATverse/.

Problem

• Limited realism and generalization in object-centric inverse rendering due to synthetic datasets
• Lack of large-scale real-world datasets with precisely controlled lighting conditions
• Insufficient comprehensive benchmarks for inverse rendering and normal estimation

Innovation

• Large-scale dataset with 9M real-world object images
• Precise lighting control using 331 individual light sources
• Multi-view capture with 35 DSLR cameras per object