Robust Camera-to-Mocap Calibration and Verification for Large-Scale Multi-Camera Data Capture

2026-04-23

AI Summary

This work addresses the challenges in extrinsic calibration between motion capture systems and external cameras—particularly fisheye cameras—in large-scale data collection, where errors arising from calibration board misalignment, ambiguous initialization, and temporal drift are difficult to detect promptly. To overcome these issues, the authors propose a robust joint calibration and independent validation framework that simultaneously estimates camera extrinsics and the transformation from the calibration board to motion capture markers. The method employs a staged nonlinear optimization strategy insensitive to initialization and introduces a fully independent validation pipeline—the “lollypop” module—that effectively handles non-uniform fisheye distortion. Experiments on the Meta Quest 3 demonstrate superior calibration accuracy over existing approaches, with the lollypop module reliably detecting calibration degradation during long-term operation. The system has been successfully deployed in real-world data acquisition pipelines.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Optical motion capture (mocap) systems are widely used for ground-truth capture in AR/VR, SLAM and robotics datasets. These datasets require extrinsic calibration to align mocap coordinates to external camera frames -- a step that is subject to multiple sources of error in practice, and failures often go undetected until they corrupt downstream data. These issues are compounded for fisheye cameras, where spatially non-uniform distortion makes both calibration and verification more challenging. We present a calibration and verification system designed for this setting. Concretely, we target robustness to board-to-marker attachment variation, optimization initialization ambiguity, and session-to-session calibration drift after deployment. The calibration jointly estimates camera extrinsics and the board-to-marker transform, and uses a staged solver to improve convergence reliability under ambiguous initialization. The verification component, \lollypop, provides fast, operator-independent assessment through a measurement chain entirely independent of the calibration data. In experiments on a Meta Quest 3 headset with fisheye cameras, our calibration outperforms existing benchwork, and lollypop reliably detects calibration degradation over time. The system has been deployed in production data collection pipelines.

Problem

• camera-to-mocap calibration
• extrinsic calibration
• fisheye cameras
• calibration verification
• multi-camera systems

Innovation

• robust calibration
• fisheye cameras
• motion capture
• extrinsic estimation
• calibration verification