Search papers, labs, and topics across Lattice.
This paper addresses the challenges of relative pose estimation with rolling shutter (RS) cameras, which are ubiquitous in consumer devices but have been hindered by the need for a large number of point correspondences. By introducing affine correspondences (ACs) and deriving RS-corrected affine constraints, the authors significantly reduce the number of required correspondences to just seven, enabling a practical and efficient linearized algebraic solver that operates in 1.2 ms. The proposed method not only achieves superior pose and RS parameter accuracy on the TUM RS benchmark but also provides accurate translational velocity estimates, outperforming existing RS solvers and demonstrating strong generalization to global-shutter scenarios.
Achieving accurate pose estimation with rolling shutter cameras using just seven correspondences could revolutionize real-time applications in consumer devices.
Rolling shutter (RS) cameras equip virtually all consumer devices, yet RS-aware relative pose estimation has remained impractical: the state-of-the-art solver requires a minimum of 20 point correspondences, making RANSAC-based robust estimation prohibitively expensive due to the exponential dependence of the iteration count on the sample size. We make RS relative pose estimation practical by introducing affine correspondences (ACs) into the RS two-view geometry. We derive novel \emph{RS-corrected affine constraints} that account for the coupling between point perturbations and the row-dependent essential matrix, providing two equations per correspondence beyond the standard epipolar constraint. Building on these constraints, we develop a linearized algebraic solver that estimates pose and RS motion from only 7 ACs. The solver exploits the physical smallness of RS parameters to linearize the constraints, eliminates the 12 RS unknowns via null-space projection, and solves the remaining degree-20 system via action matrices in 1.2\,ms. On the TUM RS benchmark, our method achieves the best pose and RS parameter accuracy among all tested methods and, uniquely among RS solvers, provides accurate translational velocity estimates -- which are poorly conditioned from point correspondences alone due to a $\vec{v}$-$\vec{t}$ coupling. On the global-shutter EuRoC MAV dataset, the solver achieves comparable accuracy to the standard 5-point algorithm, demonstrating that it generalizes well to the GS setting. Code is at https://github.com/danini/rolling_shutter_made_practical.
ETH
Google Research