Search papers, labs, and topics across Lattice.
This paper introduces CoCommit, a novel approach for improving token commitment in diffusion large language models (dLLMs) by implementing a marker-gated coordination pass that allows for joint-mode decoding. By deferring commitment and reapplying the last layers of the model to ensure coordination among selected token positions, the method effectively reduces factorization errors caused by dependent token positions. The results show significant accuracy improvements across six benchmarks, particularly in reasoning and exact-answer tasks, demonstrating the efficacy of joint commitment in enhancing model performance.
Joint commitment in dLLMs can lead to substantial accuracy gains, especially in reasoning tasks, by coordinating token selection more effectively.
Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-$n$ layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.
