Search papers, labs, and topics across Lattice.
This paper introduces Seer, a training-free framework that accelerates Diffusion Multimodal Large Language Models (DMLLMs) by detecting valid semantic boundaries at the first denoising step using a Signal-to-Noise Ratio (SNR)-based criterion. By truncating unnecessary output sequences, Seer reduces redundant computation and enhances throughput by up to 31 times without sacrificing performance. Experimental results across nine benchmarks show that Seer not only maintains but can also improve accuracy on complex visual tasks, such as increasing the DocVQA score from 63.52 to 63.66.
Truncating unnecessary output sequences at the first denoising step can boost DMLLM throughput by up to 31 times while enhancing accuracy on complex tasks.
Diffusion Multimodal Large Language Models (DMLLMs) are highly effective for multimodal reasoning, yet their inference efficiency is significantly hindered by fixed-length generation constraints. Since the actual output length is unknown, output sequences are padded to a predefined maximum length, resulting in substantial redundant computation over unnecessary [EOS] tokens. In this work, we discover that DMLLMs implicitly reveal their valid semantic boundary at the very first denoising step through a distinct shift in MLP activation sparsity. Leveraging this observation, we propose Seer, a training-free framework that detects this boundary using a Signal-to-Noise Ratio (SNR)-based criterion and performs one-shot truncation of the redundant suffix for all subsequent computations. To preserve these theoretical gains during batched serving, Seer incorporates a hybrid execution strategy that maximizes throughput while seamlessly accommodating dynamic sequence lengths. Experimental results demonstrate that Seer effectively eliminates padding waste, accelerating throughput by up to $\sim$31$\times$. Across 9 benchmarks, Seer robustly maintains overall performance and even improves accuracy on complex visual tasks by mitigating noise leakage (e.g., DocVQA score increases from 63.52 to 63.66), offering a highly efficient, plug-and-play solution for DMLLM acceleration.
