Search papers, labs, and topics across Lattice.
This paper introduces TGRIP, a novel framework that enhances Bird's-Eye View (BEV) instance prediction in autonomous driving by integrating semantic guidance into the prediction process. By employing a teacher-student pipeline with Vision-Language Foundation Models, TGRIP generates dense, semantically enriched BEV maps from multi-camera images, which serve as auxiliary supervision to improve spatio-temporal representation learning. Experimental results on the nuScenes dataset show that TGRIP outperforms existing state-of-the-art models, highlighting the importance of semantic awareness in accurately forecasting object behavior in complex driving scenarios.
Integrating semantic guidance into BEV instance prediction significantly boosts performance, proving that understanding object-specific behavior is crucial for autonomous driving.
Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles. This absence of explicit semantic awareness imposes limitations on the capacity of the model to solve ambiguities in complex scenarios, particularly those where object-specific behavior is essential for accurate forecasting (e.g. overtaking, intersections). In this paper, we introduce Text-Guided Representation for Instance Prediction (TGRIP), a novel framework that bridges this gap by injecting rich semantic priors into the instance prediction loop. The proposed teacher-student pipeline employs Vision-Language Foundation Models to generate dense, semantic-enhanced BEV maps from multi-camera images. These maps serve as auxiliary supervision during training, guiding the network to learn spatio-temporal representations that are not only geometrically consistent but also semantically discriminative. To the best of our knowledge, this represents the first attempt to unify semantic guidance with the temporal task of future instance prediction. The experimental results demonstrate that TGRIP surpasses existing state-of-the-art models in nuScenes, validating the hypothesis that semantic enrichment is a fundamental element for robust, end-to-end motion prediction. Code is available on https://github.com/miguelag99/TGRIP.
