Search papers, labs, and topics across Lattice.
This paper introduces nested sequential Monte Carlo (NSMC) methods for inference-time control in discrete diffusion language models, aiming to enhance text generation by steering sampling towards sequence-level rewards without the need for retraining. The authors identify and rectify biases in prior formulations that lead to inaccurate estimates, demonstrating that their approach effectively mitigates issues like overoptimism and weight degeneracy found in traditional particle-based methods. Evaluations on toxicity and fluency steering tasks reveal that both NSMC and fully-adapted NSMC (FA-NSMC) significantly outperform existing techniques such as best-of-$n$ sampling and bootstrap SMC.
Nested sequential Monte Carlo methods can dramatically improve inference-time control in text generation, outperforming traditional techniques in steering towards desired outcomes.
We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining. Prior work in this domain has focused on particle-based methods such as best-of-$n$ sampling and bootstrap sequential Monte Carlo, which may suffer from overoptimism and weight degeneracy, respectively. We address these limitations using \emph{nested} sequential Monte Carlo methods. We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering, identifying and correcting errors in prior formulations that lead to biased final estimates. We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-$n$ and bootstrap SMC.
