RoboLight: A Dataset with Linearly Composable Illumination for Robotic Manipulation

2026-03-04

AI Summary

This work addresses the lack of realistic, systematically varied, and scalable illumination conditions in existing robotic manipulation datasets, which limits the robustness of perception models under complex lighting. To bridge this gap, the authors present the first real-world robotic manipulation dataset with synchronized, structurally controlled illumination—spanning color, direction, and intensity—captured using programmable RGB LED lighting and high dynamic range (HDR) imaging. Leveraging the principle of linear light transport, they propose an HDR interpolation-based synthesis method to enable data augmentation at arbitrary granularity. The released dataset comprises 2,800 real-world clips and 196,000 synthetically generated clips, demonstrating high diversity, task difficulty, and the effectiveness of synthetic data. Both software and hardware systems are open-sourced to support future research.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Computational Photography, Image & Video Synthesis
• Humans and AI: Interaction Techniques and Devices

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

In this paper, we introduce RoboLight, the first real-world robotic manipulation dataset capturing synchronized episodes under systematically varied lighting conditions. RoboLight consists of two components. (a) RoboLight-Real contains 2,800 real-world episodes collected in our custom Light Cube setup, a calibrated system equipped with eight programmable RGB LED lights. It includes structured illumination variation along three independently controlled dimensions: color, direction, and intensity. Each dimension is paired with a dedicated task featuring objects of diverse geometries and materials to induce perceptual challenges. All image data are recorded in high-dynamic-range (HDR) format to preserve radiometric accuracy. Leveraging the linearity of light transport, we introduce (b) RoboLight-Synthetic, comprising 196,000 episodes synthesized through interpolation in the HDR image space of RoboLight-Real. In principle, RoboLight-Synthetic can be arbitrarily expanded by refining the interpolation granularity. We further verify the dataset quality through qualitative analysis and real-world policy roll-outs, analyzing task difficulty, distributional diversity, and the effectiveness of synthesized data. We additionally demonstrate three representative use cases of the proposed dataset. The full dataset, along with the system software and hardware design, will be released as open-source to support continued research.

Problem

• robotic manipulation
• illumination variation
• real-world dataset
• HDR imaging
• lighting conditions

Innovation

• linearly composable illumination
• HDR robotic dataset
• programmable lighting system
• synthetic data interpolation
• radiometric accuracy