Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ADAPT, a novel multimodal framework designed for pedestrian crossing intention prediction that effectively integrates local and global visual contexts with temporal motion dynamics. By leveraging a combination of RGB images, depth maps, semantic maps, and motion features through specialized modules, ADAPT achieves superior performance on benchmark datasets while minimizing computational overhead. The results show significant improvements over existing methods, with AUC scores of 0.73 and 0.85 on JAAD, and an impressive accuracy of 0.92 on PIE, all while maintaining real-time inference capabilities.
ADAPT achieves state-of-the-art pedestrian crossing intention prediction with a remarkable balance of accuracy and real-time performance, outperforming existing methods while processing multimodal inputs efficiently.
Predicting pedestrian crossing intention is a safety-critical task for autonomous driving, yet existing approaches often rely on single-modal inputs or dense multimodal fusion strategies that inadequately capture complementary visual and kinematic information while introducing redundant inter-modal interactions. We propose ADAPT (Adaptive Domain-Aware Pedestrian Crossing Transformer), a multimodal framework that jointly models local and global visual context together with temporal motion dynamics for accurate pedestrian crossing intention prediction. ADAPT processes four spatially aligned visual modalities, including RGB images, local depth maps, global semantic maps, and global depth maps, together with ego-vehicle speed, pedestrian bounding boxes, and skeleton pose information through five specialized modules: a weight-shared Swin Transformer V2 backbone for visual feature extraction, a Cross-Modality Guided Attention module for hierarchical visual fusion, a Mamba-based Motion Feature Encoding module for efficient temporal modeling, a Sparse Cross-Modal Attention module that selectively preserves the most informative inter-modal interactions, and a Vision Transformer-based Temporal Feature Fusion module for sequence-level prediction. Extensive experiments on the JAAD and PIE benchmark datasets demonstrate that ADAPT consistently outperforms existing state-of-the-art methods while maintaining low computational complexity. On JAAD, the proposed method achieves an AUC of 0.73 on JAADbeh and 0.85 on JAADall, while on PIE it achieves an accuracy of 0.92 and an AUC of 0.90. Furthermore, ADAPT performs inference in only 17.23 ms per sample, offering an effective balance between predictive accuracy and real-time deployment efficiency for intelligent transportation and autonomous driving applications.