Rolling Shutter Relative Pose Estimation Made Practical

2026-06-25

AI Summary

This work addresses the high computational cost of RANSAC in relative pose estimation with consumer-grade rolling shutter cameras, which typically requires at least 20 point correspondences. For the first time, affine correspondences are introduced into rolling shutter geometry, leading to a novel rolling shutter-corrected affine constraint that jointly incorporates row-wise dependent essential matrices and point perturbations. This formulation enables linear pose and rolling shutter motion parameter estimation from merely seven correspondences. By projecting into the nullspace, twelve unknowns are eliminated, and an efficient solver based on action matrices resolves the resulting 20th-degree polynomial system in just 1.2 ms. Evaluated on the TUM RS benchmark, the method achieves state-of-the-art accuracy in both pose and rolling shutter parameters—uniquely enabling accurate translational velocity estimation—and matches the precision of the classic five-point algorithm on the EuRoC MAV dataset.

Abstract

Rolling shutter (RS) cameras equip virtually all consumer devices, yet RS-aware relative pose estimation has remained impractical: the state-of-the-art solver requires a minimum of 20 point correspondences, making RANSAC-based robust estimation prohibitively expensive due to the exponential dependence of the iteration count on the sample size. We make RS relative pose estimation practical by introducing affine correspondences (ACs) into the RS two-view geometry. We derive novel \emph{RS-corrected affine constraints} that account for the coupling between point perturbations and the row-dependent essential matrix, providing two equations per correspondence beyond the standard epipolar constraint. Building on these constraints, we develop a linearized algebraic solver that estimates pose and RS motion from only 7 ACs. The solver exploits the physical smallness of RS parameters to linearize the constraints, eliminates the 12 RS unknowns via null-space projection, and solves the remaining degree-20 system via action matrices in 1.2\,ms. On the TUM RS benchmark, our method achieves the best pose and RS parameter accuracy among all tested methods and, uniquely among RS solvers, provides accurate translational velocity estimates -- which are poorly conditioned from point correspondences alone due to a $\vec{v}$-$\vec{t}$ coupling. On the global-shutter EuRoC MAV dataset, the solver achieves comparable accuracy to the standard 5-point algorithm, demonstrating that it generalizes well to the GS setting. Code is at https://github.com/danini/rolling_shutter_made_practical.

Problem

• Rolling Shutter
• Relative Pose Estimation
• RANSAC
• Point Correspondences
• Computational Efficiency

Innovation

• rolling shutter
• affine correspondences
• relative pose estimation
• algebraic solver
• null-space projection