Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Fenchel Tilt Flow Control (FTFC) decouples utility optimization from generative model fitting by jointly learning an effective reward and density-ratio weights on pretrained samples, leveraging Fenchel duality to transform general $f$-divergence penalties into distribution-correction weights. These weights are then used in a single stage of importance-weighted denoising or flow matching to adapt diffusion or flow models without differentiating through sampling trajectories. FTFC achieves superior performance across image and molecule generation benchmarks and is up to 20x more efficient than existing methods, enabling adaptation to diverse preferences beyond simple expected-reward maximization.
A novel Fenchel duality approach allows efficient generative model adaptation to arbitrary utility functions, achieving up to 20x speedup and broader preference support than existing methods.
Adapting a pretrained generative model to an arbitrary preference expressed as a utility function underlies reward alignment, guided design, and constraint satisfaction, enabling diverse applications. Existing fine-tuning methods trade off generality against computational cost: they either restrict the family class of supported preferences to keep optimization simple or preserve generality at the expense of efficiency. We introduce Fenchel Tilt Flow Control (FTFC), which decouples utility optimization from generative-model fitting. FTFC first optimizes for a target distribution by jointly fitting an effective reward and density-ratio weights on pretrained samples. Method combines the utility's variational structure with Fenchel duality, supporting general $f$-divergence penalties that determine how rewards are transformed into an distribution-correction weights. These weights are then frozen and used to modify a diffusion or flow model in a single stage of importance-weighted denoising or flow matching, without differentiating through sampling trajectories. We establish exact duality for concave utilities under suitable conditions and show that weighted fitting reproduces the optimal target distribution for a given utility. Across image and molecule generation benchmarks, FTFC improves over baselines on diverse preference functions, while also being up to $20\times$ more efficient. roposed method enables adaptation beyond expected-reward maximization without complex optimization, while preserving robustness for more general class of the utility functions compared to baselines.