Search papers, labs, and topics across Lattice.
This paper introduces Alignment-Guided Score Matching (AGSM), a reward-free post-training method that refines soft text tokens by integrating contrastive alignment guidance directly into the score-matching objective of diffusion models. AGSM addresses the over-penalization of negative pairs in contrastive learning, which can lead to issues like over-counting and repetition in generated images. Experiments demonstrate that AGSM matches the performance of SoftREPA while significantly improving counting accuracy by over 35% on the GenEval benchmark and is applicable to various diffusion backbones.
By injecting alignment guidance directly into the diffusion model's score function, AGSM avoids the pitfalls of contrastive learning and generates images with improved counting accuracy and semantic fidelity.
Diffusion models generate highly realistic images but often struggle with precise text-image alignment. While recent post-training methods improve alignment using external rewards or human preference signals, their performance heavily depends on reward quality and does not directly address alignment within the diffusion process itself. Recent reward-free approaches such as SoftREPA demonstrate that optimizing soft text tokens via contrastive learning can effectively improve text-image representation alignment, outperforming standard parameter-efficient fine-tuning baselines. However, the contrastive formulation can excessively penalize negative pairs, which manifests as characteristic failure cases such as over-counting and repetition. To address this issue, we propose a lightweight, reward-free post-training method that refines soft tokens by integrating contrastive alignment guidance directly into the score-matching objective of diffusion models. By assigning alignment directions at the score level, our approach mitigates these limitations and yields more coherent and semantically faithful generations. Experiments show that our method matches SoftREPA while substantially improving its failure cases, achieving over 35% improvement in counting accuracy on the GenEval benchmark. Our method is seamlessly applicable to existing diffusion backbones (SD1.5, SDXL, and SD3), and is complementary to existing RL-based diffusion post-training methods. Project page: https://jaayeon.github.io/AGSM
