Search papers, labs, and topics across Lattice.
This paper introduces Histogram-constrained Image Generation (HIG), a novel control mechanism for diffusion models that enforces user-specified distributional constraints during the image generation process. By modeling the control as an optimal transport problem, HIG achieves a balance between global coherence and local precision, allowing for precise adherence to constraints such as color histograms. The results demonstrate HIG's versatility in various applications, including constrained generation and high-capacity information embedding, highlighting its potential to enhance controllability in generative modeling.
Achieving precise control over image generation with user-defined distributional constraints could redefine how we approach generative modeling.
Diffusion models have emerged as a dominant paradigm in generative modeling, enabling high-fidelity sampling from complex data distributions. Despite impressive capabilities, controlling diffusion models to produce outputs aligned with user intent remains an open challenge, especially when balancing global coherence with local precision. Existing control mechanisms vary in the granularity of their conditioning signals. For example, textual prompts guide generation globally through high-level semantics, while ControlNet-like approaches secure precise local structure via dense conditions. In this work, we introduce Histogram-constrained Image Generation (HIG), a novel control mechanism that falls into the middle ground of control granularity. Our framework enforces user-specified distributional constraints (e.g., color histograms or latent token distributions) during the generation process with exact precision. We model such control as an optimal transport (OT) problem and apply explicit guidance transformations during sampling, thereby driving the diffusion trajectory to align with the desired histogram. We demonstrate the versatility of HIG across diverse applications, including constrained generation via color/latent histograms and high-capacity information embedding through histogram-level encoding. Our findings underscore the promise of distributional control, a flexible and interpretable control scheme that is fully compatible with existing control mechanisms, diversifying the hybrid strategies for controllable image generation. Our project page is available at: https://maps-research.github.io/hig/.
