Search papers, labs, and topics across Lattice.
ParetoSlider is introduced as a multi-objective RL framework that trains a single diffusion model to approximate the Pareto front for multiple, potentially conflicting, reward criteria. The model is trained with continuously varying preference weights as a conditioning signal, enabling inference-time navigation of optimal trade-offs. Experiments across SD3.5, FluxKontext, and LTX-2 show that a single ParetoSlider model matches or exceeds the performance of individually trained models for fixed reward trade-offs, while providing fine-grained control.
Ditch the fixed trade-offs: ParetoSlider lets you smoothly navigate competing generative goals in diffusion models at inference time, without retraining.
Reinforcement Learning (RL) post-training has become the standard for aligning generative models with human preferences, yet most methods rely on a single scalar reward. When multiple criteria matter, the prevailing practice of ``early scalarization'' collapses rewards into a fixed weighted sum. This commits the model to a single trade-off point at training time, providing no inference-time control over inherently conflicting goals -- such as prompt adherence versus source fidelity in image editing. We introduce ParetoSlider, a multi-objective RL (MORL) framework that trains a single diffusion model to approximate the entire Pareto front. By training the model with continuously varying preference weights as a conditioning signal, we enable users to navigate optimal trade-offs at inference time without retraining or maintaining multiple checkpoints. We evaluate ParetoSlider across three state-of-the-art flow-matching backbones: SD3.5, FluxKontext, and LTX-2. Our single preference-conditioned model matches or exceeds the performance of baselines trained separately for fixed reward trade-offs, while uniquely providing fine-grained control over competing generative goals.
