LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions

2026-02-01

AI Summary

This work addresses the challenges of inverse rendering in urban scenes, where complex lighting conditions—including multiple light sources, indirect illumination, and shadows—hinder the disentanglement of intrinsic scene properties and accurate 3D reconstruction, compounded by the absence of high-quality datasets. To bridge this gap, we introduce LightCity, a high-fidelity synthetic urban dataset comprising over 50,000 images captured under more than 300 sky illumination conditions, encompassing both street-level and aerial viewpoints. LightCity provides physically accurate ground-truth annotations for depth, surface normals, material properties, and direct/indirect lighting components. As the first dataset to systematically support inverse rendering research under diverse lighting scenarios in urban environments, it establishes a reliable benchmark for three fundamental tasks, thereby advancing the field of urban inverse rendering and 3D reconstruction.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Humans and AI: Crowd Sourcing and Human Computation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Inverse rendering in urban scenes is pivotal for applications like autonomous driving and digital twins. Yet, it faces significant challenges due to complex illumination conditions, including multi-illumination and indirect light and shadow effects. However, the effects of these challenges on intrinsic decomposition and 3D reconstruction have not been explored due to the lack of appropriate datasets. In this paper, we present LightCity, a novel high-quality synthetic urban dataset featuring diverse illumination conditions with realistic indirect light and shadow effects. LightCity encompasses over 300 sky maps with highly controllable illumination, varying scales with street-level and aerial perspectives over 50K images, and rich properties such as depth, normal, material components, light and indirect light, etc. Besides, we leverage LightCity to benchmark three fundamental tasks in the urban environments and conduct a comprehensive analysis of these benchmarks, laying a robust foundation for advancing related research.

Problem

• inverse rendering
• multi-illumination
• urban scenes
• 3D reconstruction
• intrinsic decomposition

Innovation

• inverse rendering
• multi-illumination
• indirect lighting
• urban dataset
• 3D reconstruction