Search papers, labs, and topics across Lattice.
This paper introduces RGB-HS, a novel framework that enhances depth estimation from thermal images by leveraging hierarchical supervision from RGB-based foundation models. By integrating a parallel RGB branch and aligning tokens across multiple levels, the method captures both structural precision and semantic abstraction, optimizing the use of rich representations encoded in RGB models. Experimental results show that RGB-HS outperforms existing approaches, effectively utilizing the representational capacity of RGB models for improved depth estimation in challenging conditions.
Hierarchical supervision from RGB models can significantly boost depth estimation accuracy in thermal imaging, even in adverse conditions.
Depth estimation from thermal images is highly valuable for robotic applications in adverse conditions, such as nighttime and rainy weather. Recent studies have sought to transfer knowledge from RGB-based foundation models to thermal modalities, yet the rich hierarchical representations these models encode remain underutilized. To address this limitation, we propose RGB-HS, a novel framework for thermal-image depth estimation that leverages hierarchical supervision from an RGB-based foundation model. Specifically, we first replace the baseline thermal encoder with a foundational model and introduce a parallel RGB branch that also employs a foundational model as an encoder of the same architecture, taking RGB images as input. The alignment is then performed across multiple levels between the tokens of the two encoders, allowing the thermal student branch to capture both structural precision and semantic abstraction from the RGB teacher branch. Furthermore, we introduce verification to refine the alignment process by weighting tokens from the RGB branch based on RGB image quality. Extensive experiments on the popular benchmark demonstrate that RGB-HS achieves competitive performance and more effectively exploits the representational capacity of RGB-based foundation models for depth estimation on thermal images.
