FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases

2025-09-05

AI Summary

To address the challenge of training optical flow estimation models under hardware resource constraints, this paper proposes FlowSeek—a lightweight framework that synergistically integrates a single-image depth foundation model with a low-dimensional motion basis parameterization, enabling joint optimization of feature learning and motion modeling. FlowSeek employs a compact network architecture, depth-guided feature extraction, and low-rank decomposition of motion bases to significantly reduce computational overhead. Compared to SEA-RAFT, FlowSeek achieves 10% and 15% lower endpoint errors on the Sintel Final and KITTI benchmarks, respectively, and attains state-of-the-art performance on cross-domain datasets including Spring and LayeredFlow. Crucially, FlowSeek reduces training hardware requirements by an order of magnitude (8×), markedly improving generalizability and deployment efficiency for resource-constrained environments.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Learning to Search
• Machine Learning: Hardware-aware ML

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We present FlowSeek, a novel framework for optical flow requiring minimal hardware resources for training. FlowSeek marries the latest advances on the design space of optical flow networks with cutting-edge single-image depth foundation models and classical low-dimensional motion parametrization, implementing a compact, yet accurate architecture. FlowSeek is trained on a single consumer-grade GPU, a hardware budget about 8x lower compared to most recent methods, and still achieves superior cross-dataset generalization on Sintel Final and KITTI, with a relative improvement of 10 and 15% over the previous state-of-the-art SEA-RAFT, as well as on Spring and LayeredFlow datasets.

Problem

• Reducing hardware resources for optical flow training
• Improving cross-dataset generalization accuracy
• Combining depth models with motion parametrization

Innovation

• Leverages depth foundation models for optical flow
• Uses low-dimensional motion parametrization for efficiency
• Trains on single consumer GPU with compact architecture