Search papers, labs, and topics across Lattice.
This paper addresses the challenge of estimating continuous optical flow from event-based cameras, which offer high temporal resolution but lack dense ground truth. They introduce a hybrid-supervised framework based on Spatio-temporal Structural Consistency (STSC) that enforces both local structural stability and trajectory continuity. The method uses a bidirectionally complementary multi-scale architecture and curriculum-guided training to achieve state-of-the-art performance on multiple benchmarks.
Enforcing spatio-temporal structural consistency in event-based optical flow estimation yields state-of-the-art performance, even surpassing traditional frame-based methods.
Estimating continuous optical flow is a fundamental yet challenging problem in dynamic visual perception. Event-based cameras, with microsecond latency and high dynamic range, capture brightness changes asynchronously, offering a unique opportunity to model motion with fine temporal precision. However, the scarcity of temporally dense ground-truth annotations limits the effectiveness of supervised learning, while contrast maximization (CM) frameworks, focused on sharpening the Image of Warped Events (IWE), often neglect temporal continuity and structural coherence, leading to distorted trajectories under complex motion. To overcome these challenges, we propose a hybrid-supervised framework for continuous-time optical flow estimation, grounded in the principle of Spatio-temporal Structural Consistency (STSC). This paradigm jointly enforces local structural stability and trajectory continuity, ensuring physically coherent motion across time. To further enhance representation and robustness, we design a bidirectionally complementary multi-scale architecture and employ a curriculum-guided hybrid training strategy, enabling a smooth transition from supervised point constraints to self-supervised manifold regularization. Comprehensive experiments across multiple benchmarks show that our method achieves state-of-the-art performance in both continuous-time and standard optical flow estimation, demonstrating the effectiveness of the proposed learning paradigm.
