Search papers, labs, and topics across Lattice.
This paper introduces ST-Prune, a training-free token pruning framework for VLMs in autonomous driving that leverages spatio-temporal redundancies in multi-view camera and multi-frame video input. It uses Motion-aware Temporal Pruning (MTP) to prioritize dynamic trajectories and current-frame content, and Ring-view Spatial Pruning (RSP) to eliminate duplicate projections across camera views. Experiments across perception, prediction, and planning benchmarks show that ST-Prune achieves near-lossless performance with up to 90% token reduction, sometimes even surpassing the full model baseline.
Achieve near-lossless performance in autonomous driving VLMs with 90% token reduction – without any training.
Vision-Language Models (VLMs) have become central to autonomous driving systems, yet their deployment is severely bottlenecked by the massive computational overhead of multi-view camera and multi-frame video input. Existing token pruning methods, primarily designed for single-image inputs, treat each frame or view in isolation and thus fail to exploit the inherent spatio-temporal redundancies in driving scenarios. To bridge this gap, we propose ST-Prune, a training-free, plug-and-play framework comprising two complementary modules: Motion-aware Temporal Pruning (MTP) and Ring-view Spatial Pruning (RSP). MTP addresses temporal redundancy by encoding motion volatility and temporal recency as soft constraints within the diversity selection objective, prioritizing dynamic trajectories and current-frame content over static historical background. RSP further resolves spatial redundancy by exploiting the ring-view camera geometry to penalize bilateral cross-view similarity, eliminating duplicate projections and residual background that temporal pruning alone cannot suppress. These two modules together constitute a complete spatio-temporal pruning process, preserving key scene information under strict compression. Validated across four benchmarks spanning perception, prediction, and planning, ST-Prune establishes new state-of-the-art for training-free token pruning. Notably, even at 90\% token reduction, ST-Prune achieves near-lossless performance with certain metrics surpassing the full-model baseline, while maintaining inference speeds comparable to existing pruning approaches.
