Double Deep Learning-based Event Data Coding and Classification

2024-07-22

arXiv.org

AI Summary

To address the high bandwidth and storage overheads associated with massive asynchronous event streams from event cameras, this paper proposes an end-to-end jointly optimized framework. It adopts point clouds as a unified representation and establishes bidirectional differentiable mappings between events and point clouds, enabling co-training of compression encoding and downstream classification tasks. This work is the first to introduce learned point cloud compression—specifically JPEG Pleno Point Cloud Compression (PCC)—to event data compression, and empirically validates the feasibility of direct classification in the compressed domain, thereby significantly reducing decoding overhead and distortion accumulation. Experiments demonstrate that, while preserving the original event-based classification accuracy, the proposed method achieves higher compression ratios and superior classification accuracy compared to conventional MPEG Geometry-based PCC (G-PCC). These results substantiate the effectiveness and advancement of intelligent processing directly within the compressed domain.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Data Mining & Knowledge Management: Data Compression

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Event cameras have the ability to capture asynchronous per-pixel brightness changes, called"events", offering advantages over traditional frame-based cameras for computer vision applications. Efficiently coding event data is critical for transmission and storage, given the significant volume of events. This paper proposes a novel double deep learning-based architecture for both event data coding and classification, using a point cloud-based representation for events. In this context, the conversions from events to point clouds and back to events are key steps in the proposed solution, and therefore its impact is evaluated in terms of compression and classification performance. Experimental results show that it is possible to achieve a classification performance of compressed events which is similar to one of the original events, even after applying a lossy point cloud codec, notably the recent learning-based JPEG Pleno Point Cloud Coding standard, with a clear rate reduction. Experimental results also demonstrate that events coded using JPEG PCC achieve better classification performance than those coded using the conventional lossy MPEG Geometry-based Point Cloud Coding standard. Furthermore, the adoption of learning-based coding offers high potential for performing computer vision tasks in the compressed domain, which allows skipping the decoding stage while mitigating the impact of coding artifacts.

Problem

• Efficient coding of high-volume event camera data
• Accurate event classification after lossy compression
• Comparing learning-based vs traditional point cloud coding standards

Innovation

• Double deep learning for event coding and classification
• Point cloud-based representation for event data
• JPEG Pleno Point Cloud Coding for compression