Search papers, labs, and topics across Lattice.
This study investigates the commitment order in masked diffusion language models (dLLMs) and its impact on reasoning tasks, revealing that unconstrained decoding leads to premature answer commitments in 15-24% of cases while significant reasoning remains masked. The authors demonstrate that this issue arises from reachability rather than termination beliefs, and they introduce a 2x2 prompt-decoder design that shows chain-of-thought reasoning significantly improves performance when commitment is ordered. By implementing a frontier-gated commitment mechanism, they achieve a substantial recovery in reasoning accuracy while maintaining efficient parallel decoding, thereby reframing existing samplers as solutions to a previously unaddressed reasoning pathology.
Unconstrained decoding in dLLMs can lead to a staggering 90% collapse into answer-only outputs, highlighting a critical flaw in reasoning capabilities.
Masked diffusion language models (dLLMs) can commit tokens in any order -- a freedom marketed as their core advantage over autoregressive decoding. We show that on reasoning tasks this freedom is instead the axis of failure. Logging every commitment during decoding of LLaDA-8B on GSM8K, we find that unconstrained (pure) decoding commits the final answer at 15-24% of the trajectory while half the reasoning region is still masked, and collapses to answer-only outputs on up to 90% of problems as the canvas grows. The cause is not the model's termination beliefs -- EOS"pressure"is nearly identical across decoders -- but reachability: whether the sampler may act on those beliefs at distant positions. A 2x2 prompt-decoder design shows that chain-of-thought helps only under ordered commitment (interaction +34.8 percentage points, 95% CI [26.8, 42.8]; without reasoning text the decoders are indistinguishable), an interaction we decompose into a collapse channel and an order channel and replicate on Dream-7B and MATH-500. A single-knob intervention -- frontier-gated commitment -- causally recovers the full gap (0.528 to 0.852) while preserving up to 4x parallel decoding, along a measured frontier whose optimal window flips from w=1 at full refinement to unconstrained at 8 tokens/step. Our results reframe existing window-style samplers, previously motivated by efficiency, as the minimal fix for a reasoning pathology they were never designed to address.
