Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces HST-HGN, a novel neural network architecture for driver fatigue assessment from video that combines hypergraph networks for spatial feature extraction with bidirectional state space models for temporal modeling. The spatial component uses a hierarchical hypergraph to capture synergistic facial deformations from pose and texture features. The temporal component employs a Bi-Mamba module for efficient bidirectional sequence modeling, enabling the system to distinguish subtle actions and capture long-range dependencies. HST-HGN achieves state-of-the-art performance on fatigue benchmarks while maintaining computational efficiency suitable for real-time edge deployment.
Achieve state-of-the-art driver fatigue detection with a hypergraph network and bidirectional state space model that runs efficiently enough for real-time in-cabin deployment.
It remains challenging to assess driver fatigue from untrimmed videos under constrained computational budgets, due to the difficulty of modeling long-range temporal dependencies in subtle facial expressions. Some existing approaches rely on computationally heavy architectures, whereas others employ traditional lightweight pairwise graph networks, despite their limited capacity to model high-order synergies and global temporal context. Therefore, we propose HST-HGN, a novel Heterogeneous Spatial-Temporal Hypergraph Network driven by Bidirectional State Space Models. Spatially, we introduce a hierarchical hypergraph network to fuse pose-disentangled geometric topologies with multi-modal texture patches dynamically. This formulation encapsulates high-order synergistic facial deformations, effectively overcoming the limitations of conventional methods. In temporal terms, a Bi-Mamba module with linear complexity is applied to perform bidirectional sequence modeling. This explicit temporal-evolution filtering enables the network to distinguish highly ambiguous transient actions, such as yawning versus speaking, while encompassing their complete physiological lifecycles. Extensive evaluations across diverse fatigue benchmarks demonstrate that HST-HGN achieves state-of-the-art performance. In particular, our method strikes a balance between discriminative power and computational efficiency, making it well-suited for real-time in-cabin edge deployment.