Globally Optimal Solution to the Generalized Relative Pose Estimation Problem using Affine Correspondences

2025-12-18

AI Summary

This paper addresses the problem of globally optimal generalized relative pose estimation for multi-camera–IMU systems under known vertical direction. The proposed method is computationally efficient and guarantees global optimality. Its core contribution lies in the first decomposition of affine correspondence geometric constraints into a univariate optimization problem involving only the rotation angle; this is achieved by constructing a bivariate polynomial system from the characteristic equation and its derivative, enabling closed-form global optimal recovery of rotation parameters. Additionally, a linear analytical solution under the small-rotation assumption is introduced to enhance computational efficiency. The approach integrates algebraic-geometric modeling, polynomial eigenvalue solving, and axis-angle parameterization, ensuring both accuracy and robustness. Evaluated on synthetic and real-world datasets, the method achieves state-of-the-art accuracy in both rotation and translation estimation, significantly outperforming existing approaches.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Motion & Tracking
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Mobile devices equipped with a multi-camera system and an inertial measurement unit (IMU) are widely used nowadays, such as self-driving cars. The task of relative pose estimation using visual and inertial information has important applications in various fields. To improve the accuracy of relative pose estimation of multi-camera systems, we propose a globally optimal solver using affine correspondences to estimate the generalized relative pose with a known vertical direction. First, a cost function about the relative rotation angle is established after decoupling the rotation matrix and translation vector, which minimizes the algebraic error of geometric constraints from affine correspondences. Then, the global optimization problem is converted into two polynomials with two unknowns based on the characteristic equation and its first derivative is zero. Finally, the relative rotation angle can be solved using the polynomial eigenvalue solver, and the translation vector can be obtained from the eigenvector. Besides, a new linear solution is proposed when the relative rotation is small. The proposed solver is evaluated on synthetic data and real-world datasets. The experiment results demonstrate that our method outperforms comparable state-of-the-art methods in accuracy.

Problem

• Estimates generalized relative pose with known vertical direction
• Uses affine correspondences for improved multi-camera system accuracy
• Provides globally optimal and linear solutions for rotation and translation

Innovation

• Globally optimal solver using affine correspondences for pose estimation
• Converts optimization into two polynomial equations with two unknowns
• Solves rotation via polynomial eigenvalue, translation from eigenvector