Search papers, labs, and topics across Lattice.
This paper introduces MMPM, a mode-aware framework for pedestrian trajectory prediction that effectively models multimodal distributions by separately addressing crossing and non-crossing behaviors. The approach incorporates a behavior-aware Pedestrian Interaction Module (PIM) that captures complex interactions through gaze, head, and hand gestures, alongside a Conditional Variational Autoencoder (CVAE)-based Mode-aware Trajectory Predictor (MTP). Experimental results on the PIE and JAAD datasets demonstrate that MMPM outperforms existing state-of-the-art methods, achieving improved accuracy in predicting realistic pedestrian movements.
MMPM reveals that accurately modeling pedestrian intentions can drastically enhance trajectory prediction, outperforming traditional methods by addressing multimodal behaviors directly.
Pedestrian trajectory prediction from an ego-centric camera is challenging since it depends on complex interactions with vehicles and scene context, as well as the intention of the pedestrian. By modelling correlation and intent from the historical and future trajectories of the pedestrian, it will usually result in a multimodal (i.e. multiple modes) distribution. Existing stochastic predictors often sample multiple futures from a single unimodal distribution, which can yield sub-optimal 'mixed-mode' trajectories that lie between distinct motion patterns and become implausible in real scenes. In this paper, we propose MMPM, a mode-aware framework that separately models future trajectory distributions into semantically meaningful modes based on the pedestrian's crossing behavior. MMPM consists of two modules: behavior-aware Pedestrian Interaction Module (PIM) that jointly captures pedestrian-vehicle and pedestrian-environment interactions by introducing gaze, head and hand gesture, and a CVAE-based Mode-aware Trajectory Predictor (MTP) module to model the future trajectory distributions on two modes, crossing and non-crossing the road, separately. A query-based decoder further enforces mode consistency during decoding. Experiments on PIE and JAAD datasets show that our method surpasses state-of-the-art baselines. Our proposed MTP is model-agnostic, which can be integrated into existing frameworks such as BiTrap-NP and SGNet-ED to further improve future trajectory prediction performance. We additionally introduce a data-driven validation protocol that matches predictions to spatio-temporally consistent ground-truth trajectories, demonstrating improved frame-wise displacement errors over previous work.
