FlowIt: Global Matching for Optical Flow with Confidence-Guided Refinement

2026-03-30

AI Summary

This work addresses the challenges of large displacements and occlusions in optical flow estimation by proposing a novel hierarchical Transformer-based framework. It introduces optimal transport theory for the first time to initialize optical flow, enabling global context modeling and generating robust initial correspondences. Subsequently, a guided refinement propagation mechanism is designed by integrating confidence maps and occlusion estimates, significantly enhancing the accuracy of long-range matching. The proposed method achieves state-of-the-art performance on the Sintel and KITTI benchmarks and demonstrates leading zero-shot cross-domain generalization capabilities across multiple datasets, including Sintel, Spring, and LayeredFlow.

Abstract

We present FlowIt, a novel architecture for optical flow estimation designed to robustly handle large pixel displacements. At its core, FlowIt leverages a hierarchical transformer architecture that captures extensive global context, enabling the model to effectively model long-range correspondences. To overcome the limitations of localized matching, we formulate the flow initialization as an optimal transport problem. This formulation yields a highly robust initial flow field, alongside explicitly derived occlusion and confidence maps. These cues are then seamlessly integrated into a guided refinement stage, where the network actively propagates reliable motion estimates from high-confidence regions into ambiguous, low-confidence areas. Extensive experiments across the Sintel, KITTI, Spring, and LayeredFlow datasets validate the efficacy of our approach. FlowIt achieves state-of-the-art results on the competitive Sintel and KITTI benchmarks, while simultaneously establishing new state-of-the-art cross-dataset zero-shot generalization performance on Sintel, Spring, and LayeredFlow.

Problem

• optical flow
• large displacement
• long-range correspondence
• occlusion
• zero-shot generalization

Innovation

• optical flow
• hierarchical transformer
• optimal transport
• confidence-guided refinement
• zero-shot generalization