An Event-Based Perception Pipeline for a Table Tennis Robot

2025-02-02

AI Summary

To address perception latency and accuracy degradation caused by motion blur in conventional frame-based cameras during high-speed table tennis, this paper proposes the first real-time, event-camera-only perception system tailored for table tennis robots. Methodologically, it employs a fully event-driven architecture integrating asynchronous event stream processing, real-time event clustering, dynamic trajectory prediction, and low-latency online filtering with motion modeling. Key contributions include: (i) the first end-to-end event-driven perception pipeline for table tennis robots; (ii) a tenfold increase in update rate; (iii) ~40% reduction in position and velocity estimation error, and 35% decrease in prediction uncertainty; and (iv) significantly enhanced robustness and precision for millisecond-level closed-loop control. The system enables reliable, high-frequency state estimation under extreme motion conditions, overcoming fundamental limitations of frame-rate-constrained vision systems.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Time-Series/Data Streams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Table tennis robots gained traction over the last years and have become a popular research challenge for control and perception algorithms. Fast and accurate ball detection is crucial for enabling a robotic arm to rally the ball back successfully. So far, most table tennis robots use conventional, frame-based cameras for the perception pipeline. However, frame-based cameras suffer from motion blur if the frame rate is not high enough for fast-moving objects. Event-based cameras, on the other hand, do not have this drawback since pixels report changes in intensity asynchronously and independently, leading to an event stream with a temporal resolution on the order of us. To the best of our knowledge, we present the first real-time perception pipeline for a table tennis robot that uses only event-based cameras. We show that compared to a frame-based pipeline, event-based perception pipelines have an update rate which is an order of magnitude higher. This is beneficial for the estimation and prediction of the ball's position, velocity, and spin, resulting in lower mean errors and uncertainties. These improvements are an advantage for the robot control, which has to be fast, given the short time a table tennis ball is flying until the robot has to hit back.

Problem

• Ping-Pong Robot
• High-Speed Tracking
• Accurate Perception

Innovation

• Event-driven Cameras
• Table Tennis Robot
• Real-time Vision System