Search papers, labs, and topics across Lattice.
The paper introduces Stitching Noisy Diffusion Thoughts, a framework that leverages diffusion language models to generate multiple reasoning trajectories, scores intermediate steps using a process reward model (PRM), and stitches together the highest-quality steps to form a composite rationale. This rationale then conditions an autoregressive model to generate the final answer, effectively separating exploration from solution synthesis. Results on math reasoning benchmarks demonstrate significant accuracy improvements (up to 23.8%) and latency reductions (up to 1.8x) compared to existing diffusion and unified architectures.
By stitching together the best reasoning steps from multiple diffusion-generated trajectories, this method achieves significant accuracy gains and latency reductions in math and coding tasks, outperforming traditional diffusion models and unified architectures.
Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from partial or"nearly correct"attempts. We propose Stitching Noisy Diffusion Thoughts, a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates. Given a problem, we (i) sample many diverse, low-cost reasoning trajectories using a masked diffusion language model, (ii) score every intermediate step with an off-the-shelf process reward model (PRM), and (iii) stitch these highest-quality steps across trajectories into a composite rationale. This rationale then conditions an autoregressive (AR) model (solver) to recompute only the final answer. This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search. Across math reasoning benchmarks, we find that step-level recombination is most beneficial on harder problems, and ablations highlight the importance of the final AR solver in converting stitched but imperfect rationales into accurate answers. Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks. At the same time, it achieves up to a 1.8x latency reduction relative to both traditional diffusion models (e.g., Dream, LLaDA) and unified architectures (e.g., TiDAR). Code is available at https://github.com/roymiles/diffusion-stitching.
