Deep Learning-based Event Data Coding: A Joint Spatiotemporal and Polarity Solution

2025-02-05

AI Summary

This work addresses the challenge of efficiently compressing massive asynchronous pixel-level event streams generated by event cameras. Conventional approaches separately model spatiotemporal coordinates and polarity, leading to representational redundancy. To overcome this, we propose the first end-to-end deep learning–driven lossy joint coding framework for event data. Our method introduces a novel single-point-cloud representation that explicitly encodes the spatiotemporal–polarity coupling, and designs a task-adaptive voxel binarization strategy to jointly optimize rate-distortion performance and downstream visual task accuracy. Experiments on event classification demonstrate that our approach achieves over 40% bitrate reduction with no accuracy degradation—significantly outperforming existing lossless and lossy event coding methods. This is the first work to empirically validate the feasibility and effectiveness of task-oriented lossy event compression, establishing a new paradigm for vision-centric event data encoding.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Neuromorphic vision sensors, commonly referred to as event cameras, have recently gained relevance for applications requiring high-speed, high dynamic range and low-latency data acquisition. Unlike traditional frame-based cameras that capture 2D images, event cameras generate a massive number of pixel-level events, composed by spatiotemporal and polarity information, with very high temporal resolution, thus demanding highly efficient coding solutions. Existing solutions focus on lossless coding of event data, assuming that no distortion is acceptable for the target use cases, mostly including computer vision tasks. One promising coding approach exploits the similarity between event data and point clouds, thus allowing to use current point cloud coding solutions to code event data, typically adopting a two-point clouds representation, one for each event polarity. This paper proposes a novel lossy Deep Learning-based Joint Event data Coding (DL-JEC) solution adopting a single-point cloud representation, thus enabling to exploit the correlation between the spatiotemporal and polarity event information. DL-JEC can achieve significant compression performance gains when compared with relevant conventional and DL-based state-of-the-art event data coding solutions. Moreover, it is shown that it is possible to use lossy event data coding with its reduced rate regarding lossless coding without compromising the target computer vision task performance, notably for event classification. The use of novel adaptive voxel binarization strategies, adapted to the target task, further enables DL-JEC to reach a superior performance.

Problem

• Efficient coding for event camera data.
• Exploiting spatiotemporal and polarity correlations.
• Lossy coding without compromising vision tasks.

Innovation

• Deep Learning-based Joint Event Coding
• Single-point cloud representation
• Adaptive voxel binarization strategies