Search papers, labs, and topics across Lattice.
This paper introduces an event-frame fusion framework for Visual Deformation Measurement (VDM) that leverages the temporal density of event data and spatial precision of frames to recover dense deformation fields. They propose an Affine Invariant Simplicial (AIS) framework that partitions the deformation field into linearized sub-regions, mitigating motion ambiguities from sparse event data using a solid elastic modeling prior. A neighborhood-greedy optimization strategy accelerates parameter searching and reduces error accumulation by propagating information from well-converged regions to their neighbors. Experiments on a newly established benchmark dataset demonstrate a 1.6% improvement in survival rate compared to state-of-the-art methods, while using significantly less data storage and processing resources than high-speed video approaches.
Achieve comparable deformation measurement accuracy with only 19% of the data storage and processing resources by fusing event streams and frames.
Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.
Tsinghua AI