Search papers, labs, and topics across Lattice.
This paper introduces NeuroPath, a novel Dual-Pathway Graph Convolutional Network designed for skeleton-based action recognition, which separately models spatial and temporal information to enhance performance. By leveraging a dual-pathway architecture inspired by human perception, the method allows for a more nuanced understanding of human motion by focusing on complementary structural and motion cues. Extensive experiments on multiple datasets, including Kinetics Skeleton 400 and NTU RGB+D, reveal significant performance improvements over traditional Spatial-Temporal Graph Convolutional Networks, highlighting the effectiveness of this approach in action recognition tasks.
Separating spatial and temporal modeling in action recognition leads to significant performance gains, challenging the effectiveness of existing implicit coupling methods.
Skeleton-based action recognition aims to recognize human actions from sequences of human joint coordinates. Most existing Spatial-Temporal Graph Convolutional Networks (STGCNs) have achieved promising results by modeling skeletal structures with implicit spatial-temporal representations. However, our empirical study reveals a clear performance imbalance across different skeletal modalities, indicating that implicitly coupling spatial and temporal information limits the full exploitation of complementary structural and motion cues. Inspired by the ventral and dorsal pathways in human perception, we propose Dual-Pathway Graph Convolutional Networks (NeuroPath), which adopt a dual-pathway architecture for separate yet collaborative modeling of spatial and temporal information. Specifically, transformation units first convert the input into pathway-specific skeletal representations, allowing each pathway to focus on complementary aspects of human motion. To further capture coordinated joint behaviors and their interrelationships, we introduce a group graph convolution block that dynamically identifies key body parts and models their spatial-temporal dependencies. In addition, inter-pathway dynamic fusion modules integrate complementary inter-modal information across pathways, facilitating higher-level semantic interpretation of actions. Extensive experiments on Kinetics Skeleton 400, NTU RGB+D 60, and NTU RGB+D 120 demonstrate consistent performance improvements, validating the effectiveness of dual-pathway spatial-temporal modeling for skeleton-based action recognition.
Tsinghua AI