Search papers, labs, and topics across Lattice.
This paper introduces VLHTrack, a novel hyperspectral object tracking framework that integrates vision and language to enhance tracking performance in dynamic scenes. By employing a Language-Guided Band Selection Module (LBSM), the model effectively reduces spectral redundancy and emphasizes discriminative features through semantic mapping. Additionally, the Dynamic Template Update with Mamba (DTUM) module enables adaptive template evolution, resulting in superior performance on the HOT2023 and HOT2024 benchmarks compared to existing state-of-the-art methods.
VLHTrack achieves unprecedented tracking accuracy by leveraging language-guided spectral selection and dynamic template updates, outperforming existing methods in challenging scenarios.
Hyperspectral object tracking (HOT) leverages the rich spectral information provided by hyperspectral videos (HSVs), offering substantial potential for object tracking. However, efficiently extracting and exploiting spectral information from redundant spectral bands remains a fundamental challenge, which severely limits model generalization and tracking performance. Moreover, in dynamic scenes, targets often experience drastic appearance variations due to factors such as occlusion and illumination changes. These variations lead to large deformations between the current frame and the template. Such discrepancies pose major challenges for existing temporal modeling approaches. In this work, we propose VLHTrack, a novel hyperspectral vision-language (VL) joint tracking framework. Specifically, we incorporate language priors to address the fundamental challenge of spectral redundancy by designing a Language-Guided Band Selection Module (LBSM). By leveraging Large Language Model (LLM) descriptions, LBSM establishes a semantic-to-spectral mapping that mitigates redundancy and accentuates discriminative spectral features. A Multi-Modal Vision-Language Fusion Module is then employed to seamlessly integrate visual and linguistic embeddings, harnessing their complementary advantages to learn coherent cross-modal representations. To address target deformation in long-term sequences, we propose a dynamic update template feature strategy implemented via the Dynamic Template Update with Mamba (DTUM) module. By leveraging selective state space modeling, DTUM learns inter-frame dependencies to update template feature, ensuring efficient template feature evolution guided by temporal context. Experiments on HOT2023 and HOT2024 demonstrate that VLHTrack outperforms state-of-the-art (SOTA) methods.
