Differential pose optimization in descriptor space -- Combining Geometric and Photometric Methods for Motion Estimation

2026-02-15

AI Summary

This work addresses the challenge of simultaneously achieving high accuracy, robustness, and loop-closure capability in two-frame pose optimization by proposing a unified framework that integrates geometric and photometric information. For the first time, dense geometric feature descriptors are incorporated into differential photometric optimization, replacing conventional photometric residuals with descriptor-based residuals to enable subpixel-level pose estimation in descriptor space. By synergistically combining the strengths of both geometric and photometric paradigms, this approach explores a novel trajectory for pose optimization grounded in descriptor similarity. Experimental results demonstrate a significant improvement in tracking accuracy; however, overall performance remains slightly inferior to reprojection error–based methods, with the primary bottleneck identified as the relatively flat landscape of descriptor similarity, which limits optimization efficacy.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

One of the fundamental problems in computer vision is the two-frame relative pose optimization problem. Primarily, two different kinds of error values are used: photometric error and re-projection error. The selection of error value is usually directly dependent on the selection of feature paradigm, photometric features, or geometric features. It is a trade-off between accuracy, robustness, and the possibility of loop closing. We investigate a third method that combines the strengths of both paradigms into a unified approach. Using densely sampled geometric feature descriptors, we replace the photometric error with a descriptor residual from a dense set of descriptors, thereby enabling the employment of sub-pixel accuracy in differential photometric methods, along with the expressiveness of the geometric feature descriptor. Experiments show that although the proposed strategy is an interesting approach that results in accurate tracking, it ultimately does not outperform pose optimization strategies based on re-projection error despite utilizing more information. We proceed to analyze the underlying reason for this discrepancy and present the hypothesis that the descriptor similarity metric is too slowly varying and does not necessarily correspond strictly to keypoint placement accuracy.

Problem

• pose optimization
• photometric error
• re-projection error
• geometric features
• descriptor space

Innovation

• descriptor residual
• differential pose optimization
• geometric-photometric fusion
• dense feature descriptors
• sub-pixel accuracy