Search papers, labs, and topics across Lattice.
Tsinghua AIChina University of Petroleum (Beijing)Heavy Oil ProcessingKey LaboratorySchool of SoftwareApr 13, 2026arXiv:2604.11140This paper introduces Hyper-FEOD, a novel object detection framework that fuses frame-based RGB data with event streams using sparse hypergraph modeling and a fine-grained mixture of experts. The Sparse Hypergraph-enhanced Cross-Modal Fusion (S-HCF) module constructs an event-guided activity map and performs hypergraph modeling on sparse tokens to capture non-local dependencies, while the Fine-Grained Mixture of Experts (FG-MoE) module uses specialized experts for different image regions. Experiments on RGB-Event benchmarks show Hyper-FEOD achieves state-of-the-art accuracy with a lightweight footprint suitable for real-time edge deployment.
Achieve state-of-the-art object detection accuracy and efficiency by fusing RGB frames and event streams with a sparse hypergraph and a fine-grained mixture of experts, enabling real-time edge deployment.
Integrating frame-based RGB cameras with event streams offers a promising solution for robust object detection under challenging dynamic conditions. However, the inherent heterogeneity and data redundancy of these modalities often lead to prohibitive computational overhead or suboptimal feature fusion. In this paper, we propose Hyper-FEOD, a high-performance and efficient detection framework, which synergistically optimizes multi-modal interaction through two core components. First, we introduce Sparse Hypergraph-enhanced Cross-Modal Fusion (S-HCF), which leverages the inherent sparsity of event streams to construct an event-guided activity map. By performing high-order hypergraph modeling exclusively on selected motion-critical sparse tokens, S-HCF captures complex non-local dependencies between RGB and event data while overcoming the traditional complexity bottlenecks of hypergraph computation. Second, we design a Fine-Grained Mixture of Experts (FG-MoE) Enhancement module to address the diverse semantic requirements of different image regions. This module employs specialized hypergraph experts tailored for object boundaries, internal textures, and backgrounds, utilizing a pixel-level spatial gating mechanism to adaptively route and enhance features. Combined with a load-balancing loss and zero-initialization strategy, FG-MoE ensures stable training and precise feature refinement without disrupting the pre-trained backbone's distribution. Experimental results on mainstream RGB-Event benchmarks demonstrate that Hyper-FEOD achieves a superior accuracy-efficiency trade-off, outperforming state-of-the-art methods while maintaining a lightweight footprint suitable for real-time edge deployment.
