Search papers, labs, and topics across Lattice.
This paper introduces WorkDrive, a novel framework that enhances the performance of vision-language models (VLMs) in autonomous driving by addressing the unique challenges posed by roadwork zones. By implementing a Chain-of-Causation (CoC) annotation pipeline that integrates structured scene facts into the decision-making process, the framework enables VLMs to make more accurate trajectory predictions in environments where traditional visual cues are altered or missing. The results demonstrate a significant reduction in trajectory average displacement error (ADE) by 9.0% on the ROADWork dataset, with an additional 3.0% improvement through consistency-based reinforcement learning, showcasing the effectiveness of the proposed approach.
Roadwork zones can confuse VLMs, but WorkDrive's causal reasoning framework cuts trajectory prediction errors by up to 12% in these challenging environments.
Autonomous driving vision-language models (VLMs) struggle in roadwork zones, where familiar visual cues such as lane markings and permanent signs are altered or absent, and temporary devices such as cones and barriers redefine the drivable corridor. VLMs can detect these objects, but without explicit guidance they anchor their reasoning on familiar elements from pre-training and fail to connect work-zone observations to correct planning decisions. We propose WorkDrive, a framework that constructs perception-grounded causal reasoning for work zones and aligns it with trajectory prediction. An automated multitask perception pipeline extracts structured scene facts and injects them into a Chain-of-Causation (CoC) annotation pipeline, redirecting the annotator's attention to domain-specific elements. The resulting reasoning labels are used for supervised fine-tuning, followed by reinforcement learning with a single reward: consistency between lateral meta-actions and the predicted trajectory. On ROADWork, the largest public work-zone dataset, the proposed roadwork CoC reduces trajectory average displacement error (ADE) by 9.0\%, and consistency-based GRPO yields a further 3.0\%, achieving progressive improvement over the trajectory-only baseline. Code and data will be publicly released.
