Search papers, labs, and topics across Lattice.
This paper introduces concentrated Implicit Preference Optimization (cIPO), a novel post-training framework designed to enhance video diffusion models by addressing temporally sparse artifacts like motion collapse and object flickering. By deriving implicit preference signals from the denoising process and concentrating optimization on segments with high temporal reconstruction errors, cIPO effectively improves the perceptual realism of generated videos without the need for costly human annotations. Experimental results show that cIPO significantly enhances both video authenticity and temporal coherence across various datasets, showcasing its potential to overcome existing limitations in preference alignment for video generation.
Temporal reconstruction errors can be harnessed to significantly boost video quality, eliminating the need for expensive human annotations in preference optimization.
Recent advances in preference alignment for diffusion-based video generation, particularly via Direct Preference Optimization (DPO), have significantly improved visual quality. However, temporally sparse artifacts such as motion collapse, object flickering, and color oversaturation remain a major barrier to perceptual realism. Existing methods struggle with these issues due to two key limitations: (1) the preference attribution bottleneck, where offline human annotations are costly and fail to accurately capture learning dynamics, while online reward signals are rollout-aware but often unstable and biased; and (2) temporal credit misallocation, where uniformly applied supervision cannot effectively target the brief segments in which artifacts occur. To address these challenges, we propose concentrated Implicit Preference Optimization (cIPO), a post-training framework for video diffusion models. cIPO derives implicit preference signals directly from the denoising process: given a real video, the model adds forward noise and reconstructs it via iterative denoising, treating the original as the preferred sample and the reconstruction as the dispreferred one. This formulation captures inference-time errors without requiring human annotations or external reward models. Moreover, frame-level discrepancies between original and reconstructed videos reveal when failures occur. cIPO leverages this by computing temporal reconstruction errors and concentrating optimization on high-error segments, enabling more precise correction of failure-prone regions. Extensive experiments demonstrate that cIPO consistently enhances video authenticity and temporal coherence across multiple datasets, highlighting the effectiveness and efficiency of implicit preference with temporally concentrated optimization.
Tsinghua AI