DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework

2025-03-19

AI Summary

Existing optical flow methods are predominantly designed for low-resolution inputs and exhibit poor generalization to ultra-high-resolution videos (e.g., 8K), while lacking standardized cross-resolution evaluation benchmarks. To address this, we propose DPFlow, a dual-pyramid adaptive optical flow framework that enables zero-shot generalization from low-resolution training to 8K inference via dual-path feature extraction, scale-adaptive feature fusion, and multi-level contextual modeling. We further introduce Kubric-NK—the first synthetic high-resolution benchmark spanning 1K to 8K resolutions—generated using Kubric to ensure photorealistic motion and geometry. Leveraging Kubric, we also synthesize high-fidelity training and test data. DPFlow achieves state-of-the-art performance on MPI-Sintel, KITTI 2015, Spring, and our new 8K benchmark, significantly improving accuracy in large-displacement motion estimation and global flow consistency.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Optical flow estimation is essential for video processing tasks, such as restoration and action recognition. The quality of videos is constantly increasing, with current standards reaching 8K resolution. However, optical flow methods are usually designed for low resolution and do not generalize to large inputs due to their rigid architectures. They adopt downscaling or input tiling to reduce the input size, causing a loss of details and global information. There is also a lack of optical flow benchmarks to judge the actual performance of existing methods on high-resolution samples. Previous works only conducted qualitative high-resolution evaluations on hand-picked samples. This paper fills this gap in optical flow estimation in two ways. We propose DPFlow, an adaptive optical flow architecture capable of generalizing up to 8K resolution inputs while trained with only low-resolution samples. We also introduce Kubric-NK, a new benchmark for evaluating optical flow methods with input resolutions ranging from 1K to 8K. Our high-resolution evaluation pushes the boundaries of existing methods and reveals new insights about their generalization capabilities. Extensive experimental results show that DPFlow achieves state-of-the-art results on the MPI-Sintel, KITTI 2015, Spring, and other high-resolution benchmarks.

Problem

• Addresses optical flow estimation for high-resolution video processing.
• Proposes DPFlow for 8K resolution generalization using low-res training.
• Introduces Kubric-NK benchmark for evaluating high-res optical flow methods.

Innovation

• DPFlow: adaptive optical flow for 8K resolution
• Kubric-NK: benchmark for 1K to 8K evaluations
• State-of-the-art results on multiple benchmarks