Search papers, labs, and topics across Lattice.
This paper introduces the Dual-Correlation Hypergraph Network (DHNet) for RGB-Thermal video object detection, addressing the challenge of spatial misalignment between RGBT image pairs. By employing a Patch-based Spatial Alignment Module and a Dual Hypergraph Fusion Module, DHNet effectively captures both temporal and spatial correlations to enhance object discriminability. Experimental results on the newly created DVT-VOD1000 dataset demonstrate that DHNet achieves state-of-the-art detection accuracy, significantly advancing the field of RGBT VOD.
Achieving state-of-the-art accuracy in RGB-Thermal video object detection, DHNet tackles spatial misalignment with innovative dual-correlation learning.
RGB-Thermal (RGBT) Video Object Detection (VOD) has gained significant traction due to its ability to overcome the limitations of conventional RGB-based VOD under challenging conditions. However, spatial misalignment commonly exists between RGBT image pairs. To address this, we propose a Dual-Correlation Hypergraph Network (DHNet) that captures high-dimensional complementary information by explicitly modeling two types of correlations: temporal correlation across consecutive frames and spatial correlation from cross-modal features. Specifically, we first design a Patch-based Spatial Alignment Module (PSAM) to sequentially align the multimodal features at the local region level. Subsequently, we introduce a Dual Hypergraph Fusion Module (DHFM), which constructs separate temporal and multimodal hypergraphs to enhance object discriminability through dual-correlation learning. Furthermore, the field currently lacks a large-scale, scene-diverse benchmark dataset for comprehensive evaluation. To address this gap, we construct DVT-VOD1000, a large-scale RGBT VOD dataset containing 1,000 video sequences with 103,464 RGBT image pairs. The dataset covers diverse scenarios, including campuses, parks, transportation, rural areas, night scenes, rain, and snow. Comprehensive experiments on VT-VOD50 and our DVT-VOD1000 demonstrate that DHNet achieves state-of-the-art detection accuracy. The dataset and source code will be made publicly available on https://github.com/tzz-ahu/ to support academic research.
