Search papers, labs, and topics across Lattice.
This paper introduces a large-scale event-camera dataset for robotic table tennis, comprising over 1000 rallies recorded with synchronized high-speed cameras to facilitate real-time ball-state estimation. By leveraging this dataset, the authors train a convolutional neural network that effectively estimates the ball's position and velocity, achieving a 36% reduction in bounce-point prediction error when combined with a Kalman filter. The integration of this event-based perception system with a robotic arm enables real-time human-robot table tennis rallies, showcasing the practical application of the developed methods.
Real-time human-robot table tennis rallies are now possible, thanks to a novel event-camera dataset and advanced ball-state estimation techniques.
Robotic table tennis has emerged as a compelling benchmark for real-time robotic perception due to its fast ball dynamics and stringent timing requirements. Accurate, high-frequency, and low-latency ball state estimation is critical for reliable trajectory prediction and timely control. Traditional frame-based cameras face an inherent trade-off: low frame rates leave temporal blind spots that miss fast-moving objects and high frame rates raise data and computational cost. Event cameras instead offer microsecond temporal resolution and, under sufficient illumination, remain largely free of motion blur even at high ball speeds. However, the community lacks large-scale datasets to develop and benchmark event-based perception in realistic sports scenarios. We address this gap by introducing the first large-scale event-camera dataset for table tennis, comprising over 1000 rallies from a diverse group of players ranging from amateurs to elite-level athletes. Each recording captures the event stream alongside 14 synchronized high-speed frame-based cameras at 200 FPS, which we use to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and spin. Building on this dataset, we train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events. Treating the predicted velocity as an additional measurement in the Kalman filter reduces bounce-point prediction error by 36% relative to a position-only baseline. Finally, we close the perception-action loop by integrating the event-based system with a St盲ubli robotic arm, enabling the first real-time human-robot table tennis rallies driven by event-based perception.
