SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization

2026-06-27

AI Summary

This work addresses the challenge of poor generalization of optical flow estimation models trained on synthetic data when deployed in real-world open environments. The authors propose a network-agnostic, self-supervised domain generalization approach that introduces, for the first time, a semantic cross-domain perturbation mechanism. During training, this method injects semantic structures from real images into synthetic data, effectively blending intra-domain features with cross-domain interference. By incorporating geometric consistency constraints, the approach aligns the feature distributions between synthetic and real domains without requiring ground-truth optical flow annotations in real scenes. This enables significant improvements in model robustness under domain shift and facilitates unsupervised generalization from synthetic to real-world scenarios.

Abstract

Motions of objects and scenes carry essential intelligence in video understanding, offering rich cues for interpreting dynamic settings and interactions. Due to the cost and scarcity of high-quality annotation or ground truth of pixel-wise optical flow, however, motion estimation models are typically trained in synthetic domains while deployed in real-world domains. Addressing synthetic-to-real domain generalization challenges has been crucial for developing practical solutions in diverse open-world use cases. This paper introduces SciFlow, a simple yet effective, network-agnostic, training-based approach that leverages self-supervised learning to generalize motion estimation across synthetic and open-world domains. Specifically, SciFlow imposes semantic interference from open-world images onto synthetic images during training, blending indomain features with cross-domain interference, which enables the network to adapt to the real-world domains. Additionally, SciFlow utilizes geometric consistency to ensure validity of the self-supervision. Our experiment results show that SciFlow not only significantly enhances model robustness amidst domain variations, but also remarkably enables synthetic-to-real domain generalization without requiring any ground truth in the open world.

Problem

• optical flow
• domain generalization
• synthetic-to-real
• self-supervised learning
• motion estimation

Innovation

• domain generalization
• self-supervised learning
• optical flow
• semantic interference
• geometric consistency