Search papers, labs, and topics across Lattice.
This paper analyzes existing token compression strategies for ViT-based 3D object detectors, finding that they often degrade performance by discarding informative background cues and disrupting contextual consistency. To address this, they propose SEPatch3D, a framework that dynamically adjusts patch sizes based on spatiotemporal context and selectively refines informative coarse patches with fine-grained details. Experiments on nuScenes and Argoverse 2 demonstrate that SEPatch3D achieves significant speedups (up to 57% faster than StreamPETR) and improved efficiency compared to state-of-the-art methods, while maintaining comparable detection accuracy.
Naively compressing tokens in ViT-based 3D object detectors hurts performance, but SEPatch3D's spatiotemporally-aware dynamic patching achieves significant speedups without sacrificing accuracy.
Vision Transformer (ViT)-based sparse multi-view 3D object detectors have achieved remarkable accuracy but still suffer from high inference latency due to heavy token processing. To accelerate these models, token compression has been widely explored. However, our revisit of existing strategies, such as token pruning, merging, and patch size enlargement, reveals that they often discard informative background cues, disrupt contextual consistency, and lose fine-grained semantics, negatively affecting 3D detection. To overcome these limitations, we propose SEPatch3D, a novel framework that dynamically adjusts patch sizes while preserving critical semantic information within coarse patches. Specifically, we design Spatiotemporal-aware Patch Size Selection (SPSS) that assigns small patches to scenes containing nearby objects to preserve fine details and large patches to background-dominated scenes to reduce computation cost. To further mitigate potential detail loss, Informative Patch Selection (IPS) selects the informative patches for feature refinement, and Cross-Granularity Feature Enhancement (CGFE) injects fine-grained details into selected coarse patches, enriching semantic features. Experiments on the nuScenes and Argoverse 2 validation sets show that SEPatch3D achieves up to \textbf{57\%} faster inference than the StreamPETR baseline and \textbf{20\%} higher efficiency than the state-of-the-art ToC3D-faster, while preserving comparable detection accuracy. Code is available at https://github.com/Mingqj/SEPatch3D.
