Search papers, labs, and topics across Lattice.
The paper addresses the quality-speed trade-off in Diffusion Large Language Models (DLLMs) caused by combinatorial contradictions during parallel decoding. They introduce Rejection Mixing (ReMix), a framework that uses a Continuous Mixing State to iteratively refine token representations in a continuous space before discrete sampling, mitigating conflicts. Experiments show ReMix achieves a 2-8x inference speedup without quality degradation, demonstrating its effectiveness as a training-free method.
DLLM inference gets a 2-8x speed boost without quality loss thanks to ReMix, a training-free method that cleverly mixes continuous representations into discrete decoding.
Diffusion Large Language Models (DLLMs) promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding. This stems from the''combinatorial contradiction''phenomenon, where parallel tokens form semantically inconsistent combinations. We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency. We propose ReMix (Rejection Mixing), a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state. This intermediate state allows a token's representation to be iteratively refined in a continuous space, resolving mutual conflicts with other tokens before collapsing into a final discrete sample. Furthermore, a rejection rule reverts uncertain representations from the continuous state back to the masked state for reprocessing, ensuring stability and preventing error propagation. ReMix thus mitigates combinatorial contradictions by enabling continuous-space refinement during discrete diffusion decoding. Extensive experiments demonstrate that ReMix, as a training-free method, achieves a $2-8 \times$ inference speedup without any quality degradation.
