Event Quality Score (EQS): Assessing the Realism of Simulated Event Camera Streams via Distances in Latent Space

2025-04-16

AI Summary

To address the degradation in sim-to-real generalization caused by simulation-induced distortions in event camera data, this paper introduces the Event Quality Score (EQS)—the first differentiable and optimization-friendly metric for assessing the realism of event streams. EQS leverages features extracted by the RVT model from both real and synthetic event sequences, jointly quantifying their distributional discrepancy via cosine similarity and Wasserstein distance in the latent space, and maps this discrepancy to a scalar realism score. Systematic evaluation on the DSEC dataset demonstrates that EQS exhibits strong positive correlation (Spearman ρ > 0.92) with real-world performance on downstream detection tasks, significantly outperforming existing hand-crafted metrics. This work establishes, for the first time, a quantitative link between event stream realism and cross-domain generalization capability. The open-sourced implementation enables iterative simulator refinement and effectively bridges the event camera simulation-to-reality gap.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Event cameras promise a paradigm shift in vision sensing with their low latency, high dynamic range, and asynchronous nature of events. Unfortunately, the scarcity of high-quality labeled datasets hinders their widespread adoption in deep learning-driven computer vision. To mitigate this, several simulators have been proposed to generate synthetic event data for training models for detection and estimation tasks. However, the fundamentally different sensor design of event cameras compared to traditional frame-based cameras poses a challenge for accurate simulation. As a result, most simulated data fail to mimic data captured by real event cameras. Inspired by existing work on using deep features for image comparison, we introduce event quality score (EQS), a quality metric that utilizes activations of the RVT architecture. Through sim-to-real experiments on the DSEC driving dataset, it is shown that a higher EQS implies improved generalization to real-world data after training on simulated events. Thus, optimizing for EQS can lead to developing more realistic event camera simulators, effectively reducing the simulation gap. EQS is available at https://github.com/eventbasedvision/EQS.

Problem

• Assessing realism of simulated event camera data
• Bridging gap between synthetic and real event streams
• Improving deep learning with better event simulation

Innovation

• Uses RVT architecture activations for quality metric
• Introduces Event Quality Score (EQS) for realism assessment
• Optimizes simulators via EQS to reduce simulation gap