BiasBench: A reproducible benchmark for tuning the biases of event cameras

2025-04-25

AI Summary

Event camera bias configuration lacks standardized, real-world evaluation benchmarks, and existing simulators and test frameworks fail to support reproducible bias optimization. Method: We introduce the first real-world event-based benchmark dataset specifically designed for bias optimization, covering three distinct scenarios, grid-sampled bias parameter configurations, and their corresponding event streams. We propose an online reinforcement learning approach based on Proximal Policy Optimization (PPO) to enable adaptive, scene-aware bias adjustment. Additionally, we design downstream-task-driven quality metrics—e.g., optical flow estimation accuracy—to quantitatively assess event stream fidelity. Contribution/Results: This work establishes the first empirically grounded, automated framework for event camera bias tuning. It significantly improves event stream quality and downstream task performance, with validation conducted on physical event camera hardware, thereby bridging the gap between simulation and real-world deployment.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Event-based cameras are bio-inspired sensors that detect light changes asynchronously for each pixel. They are increasingly used in fields like computer vision and robotics because of several advantages over traditional frame-based cameras, such as high temporal resolution, low latency, and high dynamic range. As with any camera, the output's quality depends on how well the camera's settings, called biases for event-based cameras, are configured. While frame-based cameras have advanced automatic configuration algorithms, there are very few such tools for tuning these biases. A systematic testing framework would require observing the same scene with different biases, which is tricky since event cameras only generate events when there is movement. Event simulators exist, but since biases heavily depend on the electrical circuit and the pixel design, available simulators are not well suited for bias tuning. To allow reproducibility, we present BiasBench, a novel event dataset containing multiple scenes with settings sampled in a grid-like pattern. We present three different scenes, each with a quality metric of the downstream application. Additionally, we present a novel, RL-based method to facilitate online bias adjustments.

Problem

• Lack of automatic bias tuning tools for event cameras
• Need systematic testing framework for bias configuration
• Existing simulators unsuitable for bias tuning due to hardware dependencies

Innovation

• BiasBench dataset for reproducible event camera bias tuning
• Grid-like sampled settings for systematic bias evaluation
• RL-based method for online event camera bias adjustment