Search papers, labs, and topics across Lattice.
This paper investigates the use of event camera features, specifically the eigenvalues of the structure tensor and spatiotemporal density values, as motion cues for enhancing motion estimation tasks. Through theoretical analysis and controlled experiments on synthetic data, the authors demonstrate that these features provide complementary motion information, particularly in challenging conditions like texture scarcity and noise. When integrated into a state-of-the-art event-based optical flow network, the proposed features lead to significant accuracy improvements on the DSEC benchmark, especially for lower-capacity models.
Motion cues derived from event camera features can dramatically enhance optical flow accuracy, especially in data-scarce environments.
Event cameras capture intensity changes asynchronously with high temporal resolution, requiring novel preprocessing methods for downstream tasks. Unlike static intensity snapshots, event data inherently encode information about scene dynamics and object motion, meaning that features derived from events can exhibit behaviors with no direct analogue in frame-based vision. In this paper, we analyze two features used in event-based corner detection---the eigenvalues of the structure tensor and the spatiotemporal density values---and show that they are \emph{motion cues}. We hypothesize that these features, combined with local geometric information, can enhance motion estimation tasks. To validate this, we first theoretically analyze how the eigenvalues of the structure tensor at moving corner points relate to the direction of motion. We then design controlled experiments on a synthetic dataset, confirming that extending local geometric features with eigenvalues and density values provides complementary motion information and is robust to texture and shot noise. Finally, we integrate the proposed features into a state-of-the-art event-based optical flow network and evaluate on the real-world DSEC benchmark, where the added features consistently improve accuracy, with the largest gains in data-scarce scenarios and for lower-capacity models. The code for this paper can be found at: \href{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}.
