Search papers, labs, and topics across Lattice.
This paper introduces Gradient-Informed Logit Correction (GILC), a novel framework that enhances controllable generation in discrete diffusion models without the need for retraining. By leveraging a Jacobian-free mechanism to correct prediction logits, GILC effectively stabilizes guidance signals in high-dimensional spaces, accommodating various reward functions. Extensive experiments show that GILC achieves state-of-the-art performance in tasks such as DNA and protein sequence generation, often surpassing traditional fine-tuning methods.
GILC enables state-of-the-art performance in discrete diffusion models without the computational burden of retraining, revolutionizing controllable generation tasks.
Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining. In this paper, we present \underline{\textbf{G}}radient-\underline{\textbf{I}}nformed \underline{\textbf{L}}ogit \underline{\textbf{C}}orrection (\textbf{GILC}), a plug-and-play framework that efficiently estimates guidance signals by repurposing the pretrained denoising network as a variational proxy. To circumvent the gradient instability inherent in high-dimensional discrete spaces, we introduce a Jacobian-free mechanism that directly corrects the clean prediction logits, facilitating stable and effective guidance. Our method accommodates both differentiable and non-differentiable reward functions. Extensive experiments across DNA, protein sequence, and molecular generation tasks demonstrate that GILC achieves state-of-the-art performance without additional training, frequently outperforming fine-tuning approaches.
