A Real-Time Event-Based Normal Flow Estimator

2025-04-28

AI Summary

Addressing the challenge of real-time normal flow estimation for event cameras, this paper proposes an asynchronous, event-driven, lightweight approach. It models event slices as integer-coordinate 3D point clouds and introduces, for the first time, an integer-coordinate-aware differentiable pooling operation—replacing conventional adjacency matrix multiplication to overcome the O(N²) computational bottleneck. Integrated with event-driven representation learning, CUDA-accelerated MLPs, and efficient point cloud encoding, the method achieves 4.0 million and 6.0 million normal flow vectors per second on RTX 3070 and A5000 GPUs, respectively, with millisecond-level latency and only 1 GB GPU memory consumption. The full implementation—combining CUDA kernels and Python—is open-sourced.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Adversarial Attacks & Robustness
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

This paper presents a real-time, asynchronous, event-based normal flow estimator. It follows the same algorithm as Learning Normal Flow Directly From Event Neighborhoods, but with a more optimized implementation. The original method treats event slices as 3D point clouds, encodes each event's local geometry into a fixed-length vector, and uses a multi-layer perceptron to predict normal flow. It constructs representations by multiplying an adjacency matrix with a feature matrix, resulting in quadratic time complexity with respect to the number of events. In contrast, we leverage the fact that event coordinates are integers and reformulate the representation step as a pooling operation. This achieves the same effect as the adjacency matrix but with much lower computational cost. As a result, our method supports real-time normal flow prediction on event cameras. Our estimator uses 1 GB of CUDA memory and runs at 4 million normal flows per second on an RTX 3070, or 6 million per second on an RTX A5000. We release the CUDA implementation along with a Python interface at https://github.com/dhyuan99/VecKM_flow_cpp.

Problem

• Real-time normal flow estimation from event cameras
• Optimized computation reducing quadratic time complexity
• Efficient CUDA implementation for high-speed processing

Innovation

• Reformulates representation as integer-based pooling
• Optimizes CUDA for real-time normal flow
• Achieves 4-6 million flows per second