Search papers, labs, and topics across Lattice.
This paper introduces STAR-VLM, a novel framework that utilizes automotive radar supervision to enhance spatiotemporal reasoning in vision-language models (VLMs) for autonomous driving applications. By leveraging range and Doppler measurements as label-free ground truth, STAR-VLM significantly improves the models' capabilities in motion classification and metric velocity estimation, outperforming traditional task-specific methods. The findings underscore the potential of automotive radar as a scalable and cost-effective solution for advancing metric-aware spatiotemporal reasoning in real-world driving scenarios.
Automotive radar can transform VLMs by providing label-free supervision that boosts motion reasoning and metric velocity estimation capabilities.
Vision-language models (VLMs) are emerging as a key component of embodied intelligence, with growing applications in auto-labeling and end-to-end autonomous driving. However, existing approaches for improving spatiotemporal reasoning in VLMs often rely on complex preprocessing pipelines, expensive human annotations, or synthetic data, which limit scalability and introduce potential sim-to-real gaps. Moreover, although these methods have improved spatiotemporal understanding, they still lack strong metric reasoning capabilities for dynamic scenes, such as estimating object motion in real-world units. Prior work has explored LiDAR-based metric depth supervision to enhance spatial perception, but it does not directly address temporal reasoning. We introduce STAR-VLM, an automotive radar-supervised framework that enhances spatiotemporal VLMs with motion reasoning and metric velocity estimation for autonomous driving. Automotive radar is a low-cost and widely deployed sensor that provides complementary spatiotemporal supervision through range and Doppler measurements. By leveraging these measurements as label-free ground truth during training, STAR-VLM improves the metric spatiotemporal reasoning ability of VLMs. Through experiments on driving scenarios, we show that STAR-VLM achieves state-of-the-art performance on both motion classification and metric velocity estimation, outperforming even task-specific methods designed for each task. These results highlight automotive radar as a scalable and cost-effective source of supervision for building metric-aware spatiotemporal VLMs for real-world autonomous driving.
