Search papers, labs, and topics across Lattice.
This paper introduces RADIANCE, a training-free framework designed to enhance text-to-image diffusion models' ability to synthesize rare concepts by addressing compositional balance during the denoising process. By integrating a Compositional Similarity Monitor, a Bidirectional Scale Controller, and a Feedback Guidance Scheduler, RADIANCE effectively rebalances the generation trajectory to prevent concept omission and semantic drift. Experimental results on RareBench and T2I-CompBench reveal that RADIANCE significantly improves compositional alignment and perceptual quality compared to existing state-of-the-art methods.
RADIANCE boosts the success rate of synthesizing rare concepts in text-to-image models by effectively rebalancing denoising trajectories without additional training.
Text-to-image (T2I) diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional alignment and perceptual quality over state-of-the-art baselines.
