Search papers, labs, and topics across Lattice.
This paper introduces P-MTP, a novel framework that enhances document parsing efficiency by utilizing Progressive Multi-Token Prediction with a lightweight MTP module to scale look-ahead depth. By implementing Progressive Curriculum Loss, the method effectively reduces optimization instability and gradient noise, allowing for a seamless transition from easy to hard optimization tasks. Experimental results show that P-MTP achieves up to a 5x speedup in inference without significant accuracy loss, marking a significant advancement in the parsing of token-dense documents.
Achieving a 5x speedup in document parsing without sacrificing accuracy could redefine efficiency benchmarks in Vision-Language Models.
Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emerged as a promising approach for accelerating inference, its potential is constrained by optimization instability when scaling to deeper look-ahead depth. In this paper, we propose \textbf{P-MTP}, a framework that leverages \textbf{Progressive Multi-Token Prediction} with a lightweight MTP module to scale the look-ahead depth for high-throughput document parsing. Specifically, we introduce Progressive Curriculum Loss that adaptively re-weights different look-ahead depths using cumulative path reliability and retrospective target consistency. By effectively suppressing gradient noise in long-range predictions, P-MTP, facilitates an automated easy-to-hard optimization transition, enabling the model to master increasingly distant look-ahead depths. Furthermore, we propose Confidence-Gated Dynamic Drafting to maximize the effective look-ahead depth and acceptance rate by adaptively calibrating speculative length during inference, thereby minimizing computational waste and further pushing the boundaries of inference speedup. Experimental results across multiple benchmarks and architectures demonstrate that P-MTP, achieves up to a $5\times$ speedup with negligible loss in accuracy, providing the first successful validation of extensive look-ahead MTP in the document parsing domain.
Tsinghua AI