Search papers, labs, and topics across Lattice.
This study addresses the limitations of traditional hand detection algorithms by creating a synthetic dataset that combines RGB and event-based camera data for first-person hand detection. By leveraging the high temporal resolution and dynamic range of event-based cameras, the authors synthesized a multimodal dataset from the Egohands dataset, varying conditions to enhance training diversity. The results show that their approach achieves performance on par with state-of-the-art methods, significantly improving detection accuracy in challenging conditions.
Event-based cameras can dramatically enhance hand detection accuracy, even in low-light and high-motion scenarios, thanks to a novel synthetic dataset.
Existing hand detection algorithms work on images and the detection rate is restricted by the frame rate of the camera. In hand detection applications for moving robotic systems, conventional cameras cause motion blur, especially in darker lighting conditions. We can leverage the use of event-based cameras which possess a high dynamic range, high temporal resolution, and low power consumption. Recent work has shown that using a stereo setup of an event-based and a frame-based camera improves detection accuracy and the bandwidth-latency tradeoff. The main bottleneck in using event-based cameras in object detection and recognition tasks is a relatively low amount of training data. In this work, we propose a methodology and an exemplary synthetic event-based hand dataset from an egocentric, first-person view perspective. The data is synthesized from the existing RGB Egohands dataset with the v2e toolbox. Parameters of the v2e toolbox are varied to provide versions of the dataset with different lighting conditions and scales. Ground truth detections are generated with a fine-tuned YOLOv8 model which is applied to the RGB images in the Egohands dataset and interpolated on the high-temporal resolution events. We use the multi-modal dataset to perform hand detection with existing object detection algorithms which use a multi-modal setup of event and RGB cameras and demonstrate performance comparable to the state-of-the-art.
