Search papers, labs, and topics across Lattice.
This paper introduces GRAS, a method that improves training-free reward alignment in discrete diffusion models by addressing the limitations of gradient guidance and search. By employing a Rao-Blackwellized estimator to reduce variance in gradient proposals and implementing an adaptive resampling temperature, GRAS enhances the efficiency of steering models toward desired rewards without incurring additional denoiser costs. The results demonstrate that GRAS significantly outperforms existing training-free methods in tasks such as regulatory DNA and protein design, achieving rewards comparable to fine-tuned models.
GRAS achieves superior training-free reward alignment in discrete diffusion models, outperforming previous methods and rivaling fine-tuned approaches with no added computational cost.
Discrete diffusion models have become a strong, widely adopted class of generators for sequence data, and steering them toward a downstream reward at inference time, without any retraining, is increasingly important. Such training-free steering is done by gradient guidance, by search, or by combining the two. We study the combined regime and identify two weaknesses in how it is usually run: the guided proposal estimates its gradient from a single noisy sample, and the search then resamples particles at a fixed temperature that ignores how rewards spread across each denoising step. We address both with a small set of changes that add no denoiser cost. For the proposal, we lower the estimator variance with a Rao-Blackwellized reveal for differentiable rewards and a leave-one-out baseline for non-differentiable ones; for the search, we standardize the per-step values into a group-relative advantage and prove it collapses to a single active ingredient, an adaptive resampling temperature. We call the resulting method Guided Reduced-variance proposals and Adaptive Selection (GRAS). GRAS is simple yet effective: across regulatory DNA and protein design it attains the best training-free reward, outperforming prior training-free methods and matching or surpassing a reward-fine-tuned model, and it remains effective even for non-differentiable rewards.
