NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

2024-08-19

arXiv.org

AI Summary

Existing optical flow estimation methods struggle to simultaneously achieve high accuracy, computational efficiency, and cross-domain generalizability: state-of-the-art (SOTA) models incur prohibitive computational costs, whereas lightweight alternatives suffer from accuracy degradation and poor robustness across diverse scenes. To address real-time, high-accuracy requirements on edge devices, this paper proposes a lightweight yet effective optical flow architecture, centered on a minimalist CNN backbone and a cascade of fast refinement modules designed for synergistic optimization. The architecture achieves near-SOTA accuracy on benchmark datasets (e.g., Sintel and KITTI) while drastically reducing computational complexity—attaining over 20 FPS at 512×384 resolution on a Jetson Orin Nano, outperforming mainstream SOTA methods by 10–70× in speed. Extensive experiments demonstrate strong generalization across both synthetic and real-world datasets, establishing a practical, deployable solution for edge-based vision systems.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Real-time high-accuracy optical flow estimation is crucial for various real-world applications. While recent learning-based optical flow methods have achieved high accuracy, they often come with significant computational costs. In this paper, we propose a highly efficient optical flow method that balances high accuracy with reduced computational demands. Building upon NeuFlow v1, we introduce new components including a much more light-weight backbone and a fast refinement module. Both these modules help in keeping the computational demands light while providing close to state of the art accuracy. Compares to other state of the art methods, our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data. It is capable of running at over 20 FPS on 512x384 resolution images on a Jetson Orin Nano. The full training and evaluation code is available at https://github.com/neufieldrobotics/NeuFlow_v2.

Problem

• Balancing optical flow accuracy and computational efficiency
• Improving generalization in fast optical flow methods
• Achieving real-time performance on low-power devices

Innovation

• light-weight backbone for efficiency
• fast refinement module for accuracy
• real-time 20 FPS on Jetson Orin