BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation Using RGB Frames and Events

2024-10-27

European Conference on Computer Vision

AI Summary

Existing motion estimation benchmarks lack the capability to jointly evaluate multi-task models that fuse RGB frames and event data, hindering unified assessment of optical flow, scene flow, and point tracking. To address this, we propose the first RGB–event joint benchmark tailored for multi-task visual motion estimation—introducing event camera data systematically into coordinated evaluation across all three tasks. Leveraging DVS simulation and NeRF-driven dynamic scene synthesis, we construct 12 realistic and synthetic sequences encompassing five distinct motion patterns. We design a cross-modal alignment annotation strategy and render high-fidelity ground-truth motion fields, complemented by a multi-scale error metric framework. This benchmark enables fair, task-agnostic comparison across optical flow, scene flow, and point tracking, significantly enhancing the rigor of generalization evaluation under cross-modal and cross-scene settings.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Large-scale security measurements

Problem

• Benchmark for optical flow estimation
• Integration of event data and RGB images
• Comprehensive annotations for correspondence tasks

Innovation

• Combines event data with RGB images
• Provides dense per-pixel annotations
• Covers 410 everyday categories