Search papers, labs, and topics across Lattice.
This paper introduces Multi-ORFT, a method for cooperative driving that combines scene-conditioned diffusion pre-training with stable online reinforcement post-training. The pre-training stage uses inter-agent attention mechanisms and AdaLN-Zero conditioning to improve scene consistency, while the post-training stage employs a two-level MDP and variance-gated group-relative policy optimization (VG-GRPO) to stabilize training. Experiments on the WOMD benchmark demonstrate that Multi-ORFT reduces collision and off-road rates while increasing average speed compared to the pre-trained planner and other baselines.
Online reinforcement fine-tuning can measurably improve the safety and efficiency of diffusion-based multi-agent driving planners, even when starting from strong pre-trained models.
Closed-loop cooperative driving requires planners that generate realistic multimodal multi-agent trajectories while improving safety and traffic efficiency. Existing diffusion planners can model multimodal behaviors from demonstrations, but they often exhibit weak scene consistency and remain poorly aligned with closed-loop objectives; meanwhile, stable online post-training in reactive multi-agent environments remains difficult. We present Multi-ORFT, which couples scene-conditioned diffusion pre-training with stable online reinforcement post-training. In pre-training, the planner uses inter-agent self-attention, cross-attention, and AdaLN-Zero-based scene conditioning to improve scene consistency and road adherence of joint trajectories. In post-training, we formulate a two-level MDP that exposes step-wise reverse-kernel likelihoods for online optimization, and combine dense trajectory-level rewards with variance-gated group-relative policy optimization (VG-GRPO) to stabilize training. On the WOMD closed-loop benchmark, Multi-ORFT reduces collision rate from 2.04% to 1.89% and off-road rate from 1.68% to 1.36%, while increasing average speed from 8.36 to 8.61 m/s relative to the pre-trained planner, and it outperforms strong open-source baselines including SMART-large, SMART-tiny-CLSFT, and VBD on the primary safety and efficiency metrics. These results show that coupling scene-consistent denoising with stable online diffusion-policy optimization improves the reliability of closed-loop cooperative driving.
