Estimating 2D Camera Motion with Hybrid Motion Basis

2025-07-30

AI Summary

This work addresses camera motion estimation—a fundamental vision task—by proposing CamFlow, a framework for modeling camera projection dynamics under complex nonlinear transformations. Existing approaches are limited by restrictive assumptions: homography-based methods assume planar scenes, while grid-flow methods rely on local linear approximations, both failing to generalize to realistic non-rigid scenarios. To overcome these limitations, we introduce a geometry-stochastic hybrid motion basis that jointly incorporates physical constraints and stochastic basis functions. We further design a Laplacian-based mixture probabilistic loss to enhance robustness against outliers and dynamic objects. Additionally, we construct a new benchmark with dynamic-object masks for training and evaluation. Experiments demonstrate that CamFlow significantly outperforms state-of-the-art methods across diverse scenes and exhibits superior zero-shot generalization. The code and dataset are publicly released.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Estimating 2D camera motion is a fundamental computer vision task that models the projection of 3D camera movements onto the 2D image plane. Current methods rely on either homography-based approaches, limited to planar scenes, or meshflow techniques that use grid-based local homographies but struggle with complex non-linear transformations. A key insight of our work is that combining flow fields from different homographies creates motion patterns that cannot be represented by any single homography. We introduce CamFlow, a novel framework that represents camera motion using hybrid motion bases: physical bases derived from camera geometry and stochastic bases for complex scenarios. Our approach includes a hybrid probabilistic loss function based on the Laplace distribution that enhances training robustness. For evaluation, we create a new benchmark by masking dynamic objects in existing optical flow datasets to isolate pure camera motion. Experiments show CamFlow outperforms state-of-the-art methods across diverse scenarios, demonstrating superior robustness and generalization in zero-shot settings. Code and datasets are available at our project page: https://lhaippp.github.io/CamFlow/.

Problem

• Estimating 2D camera motion from 3D movements
• Overcoming limitations of homography-based and meshflow methods
• Handling complex non-linear transformations in camera motion

Innovation

• Hybrid motion bases combining physical and stochastic
• Laplace-based probabilistic loss for robust training
• Dynamic object masking for pure motion benchmark