Search papers, labs, and topics across Lattice.
EventHub is introduced as a training framework for event-based stereo networks that eliminates the need for ground truth annotations from active sensors. The framework leverages novel view synthesis techniques to generate proxy annotations and events from color images, or proxy annotations when event data is already available. This approach allows for the adaptation of existing RGB stereo models to process event data, resulting in event stereo models with improved generalization.
Ditch the expensive active sensors: EventHub lets you train high-performing event-based stereo networks using only standard color images and novel view synthesis.
We propose EventHub, a novel framework for training deep-event stereo networks without ground truth annotations from costly active sensors, relying instead on standard color images. From these images, we derive either proxy annotations and proxy events through state-of-the-art novel view synthesis techniques, or simply proxy annotations when images are already paired with event data. Using the training set generated by our data factory, we repurpose state-of-the-art stereo models from RGB literature to process event data, obtaining new event stereo models with unprecedented generalization capabilities. Experiments on widely used event stereo datasets support the effectiveness of EventHub and show how the same data distillation mechanism can improve the accuracy of RGB stereo foundation models in challenging conditions such as nighttime scenes.
