Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors develop Latent Flow Reasoning Models (LFRMs), an iterative continuous-diffusion framework that distills multi-layer autoregressive representations to enable asynchronous latent denoising and amortized prompt conditioning. Addressing the finding that latent reconstruction fidelity alone does not yield strong reasoning, the method pairs compact score-sufficient feature compression with DiffusionNFT fine-tuning augmented by gold-solution endpoints. Scaling a 638M-parameter backbone, LFRM achieves continuous-diffusion state-of-the-art results including 63.74% on GSM8K, 24.6% on MATH500 (64 steps), and 32.85% on HumanEval (128 steps).
Accurate latent reconstruction is a red herring for diffusion-based reasoning; asynchronous denoising over multi-layer teacher representations allows a compact 638M model to solve complex math and code without autoregressive rollouts.
Continuous diffusion generates complete reasoning solutions through iterative refinement in latent space. We introduce Latent Flow Reasoning Models (LFRMs), an ELF-based training and inference recipe. Our experiments show that accurate decoding alone does not ensure strong reasoning performance. We therefore learn compact representations from multiple layers of a strong autoregressive teacher. Their decomposition also enables asynchronous denoising at different rates. We show that prompt encodings need only preserve the information required for the correct text-conditional score, rather than exactly match teacher features, and use a staged curriculum to learn a compact prompt encoder that replaces the teacher Transformer at inference. We adapt DiffusionNFT to learned self-conditioning guidance and incorporate gold-solution endpoints to supplement sparse rewards. Our supervised models outperform reported results from recent continuous-diffusion baselines at comparable backbone scales on mathematical reasoning and HumanEval code generation. With a 638M-parameter denoising backbone and learned prompt conditioning, post-NFT LFRM-L achieves 63.74% pass@1 on GSM8K and 24.6% on MATH500 at 64 denoising steps, and 32.85% on HumanEval and 30.18% on HumanEval+ at 128 denoising steps. Code will be available at: https://github.com/chengxiang/LFRM