Search papers, labs, and topics across Lattice.
This paper introduces an asynchronous real-time cross-spectral visual-thermal-inertial (VTI) system that addresses the limitations of standard stereo visual-inertial odometry (VIO) by decoupling high-latency deep matching from high-rate state estimation. The proposed architecture employs a spectral-aware weighting scheme to adaptively balance reliance on visual and thermal modalities, enhancing robustness in challenging environments. Experimental results demonstrate that the system significantly improves accuracy in daylight conditions while maintaining performance in visually degraded settings, effectively mitigating the risks of spectral redundancy.
A novel cross-spectral VTI system achieves superior accuracy and robustness in challenging environments by dynamically balancing visual and thermal data reliance.
Standard stereo VIO focuses exclusively on the benefit of metric scale via single-spectrum baselines, often overlooking the risks of spectral redundancy. This structural limitation leads to correlated failures, where both sensors simultaneously fail in degraded environments that affect their shared spectrum. Leveraging a cross-spectral system presents a complementary solution to this issue, yet the significant appearance gap between modalities renders standard matching ineffective. Existing deep learning-based matchers, while effective, introduce inference latencies that violate real-time constraints. To bridge this gap, we present an asynchronous real-time cross-spectral visual-thermal-inertial (VTI) system that temporally decouples high-latency deep matching from high-rate state estimation. Our architecture incorporates a spectral-aware weighting scheme that dynamically balances modality reliance based on photometric entropy and thermal noise, ensuring robustness against both abrupt lighting changes and thermal artifacts. Furthermore, we introduce a seamless handling mechanism for thermal Non-uniformity Correction (NUC) to maintain tracking continuity. Extensive experiments across diverse scenarios confirm that our system overcomes spectral redundancy, yielding superior accuracy in nominal daylight while ensuring robustness in visually degraded environments. We will open source our code and data: https://github.com/seungsang07/cross-spectral-stereo-inertial-odometry
