Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an inference-time attention steering mechanism for vision-language-action (VLA) driving models, enabling the redirection of attention toward safety-critical traffic actors without the need for retraining. By applying a bounded additive pre-softmax attention bias on visual tokens localized to traffic actors, the authors demonstrate significant improvements in trajectory decoding, achieving a mean displacement of approximately 17 cm with lateral shifts up to 140 cm. The findings reveal that the attention bias effectively influences the model's focus, steering trajectories toward attended actors while maintaining the integrity of the reasoning pathway.
Steering attention at inference time can drastically improve safety in VLA driving models without retraining, achieving up to 140 cm lateral shifts in trajectory predictions.
Vision-language-action (VLA) driving models couple a reasoning stage with a diffusion-based trajectory decoder, but do not give a direct way to redirect attention toward safety-critical actors at inference time without retraining. We studied a bounded additive pre-softmax attention bias on the visual tokens of detector localized traffic actors on Alpamayo-R1's Qwen3-VL backbone. It is applied as a fail open forward pre-hook with no weight changes. On 50 lane-change scenarios from the Physical AI World Model Synthetic dataset. The trajectory decoder shows a monotonic dose response in the bias magnitude, separate from a paired zero bias control at every tested magnitude. It reaches $\approx 17$\,cm mean displacement with lateral shifts up to $\sim 140$\ cm at the clamp. A layer ablation places the action-relevant signal in late layers, where the effect increases with the number of hooked layers (2.0cm for the first 8 layers; 67.6cm for all 36). A per call injection audit explains why the Chain-of-Causation text never changes. The mask based bias never reaches the reasoning pathway in this serving stack, so the invariance is verified exposure, not robustness. Steered trajectories tend to shift toward the attended actor, suggesting the bias governs where the model looks rather than encoding a target behavior.