LiFMCR: Dataset and Benchmark for Light Field Multi-Camera Registration

2025-10-15

AI Summary

Existing light field datasets are predominantly captured with single-camera setups and lack external ground truth, hindering rigorous evaluation of multi-camera light field registration. To address this, we introduce the first synchronized dual-microlens-array light field camera dataset, equipped with high-precision 6-DoF pose ground truth obtained from a Vicon motion capture system. We further propose the first benchmark for multi-light-field-camera registration, unifying point-cloud registration and light field PnP paradigms while explicitly incorporating the light field camera model. Our method employs RANSAC-optimized 3D transformation estimation and a customized light field PnP algorithm, achieving high-accuracy cross-view registration on the Raytrix R32 dual-camera array. Experiments demonstrate significantly reduced registration error relative to ground truth, substantially improving the reliability of multi-view light field alignment and fusion.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We present LiFMCR, a novel dataset for the registration of multiple micro lens array (MLA)-based light field cameras. While existing light field datasets are limited to single-camera setups and typically lack external ground truth, LiFMCR provides synchronized image sequences from two high-resolution Raytrix R32 plenoptic cameras, together with high-precision 6-degrees of freedom (DoF) poses recorded by a Vicon motion capture system. This unique combination enables rigorous evaluation of multi-camera light field registration methods. As a baseline, we provide two complementary registration approaches: a robust 3D transformation estimation via a RANSAC-based method using cross-view point clouds, and a plenoptic PnP algorithm estimating extrinsic 6-DoF poses from single light field images. Both explicitly integrate the plenoptic camera model, enabling accurate and scalable multi-camera registration. Experiments show strong alignment with the ground truth, supporting reliable multi-view light field processing. Project page: https://lifmcr.github.io/

Problem

• Develops dataset for multi-camera light field registration
• Provides synchronized multi-view images with ground truth
• Enables evaluation of plenoptic camera registration methods

Innovation

• Dataset with synchronized multi-camera light field images
• RANSAC-based 3D transformation using cross-view point clouds
• Plenoptic PnP algorithm estimating poses from single images