Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper investigates how to improve style transfer in pre-trained diffusion models by optimizing the location and strength of style injection. They systematically explore varying style injection strength across decoder layers and denoising timesteps, as well as scheduling ControlNet geometric conditioning. The key finding is that decreasing schedules, with stronger structural signal injection in shallower layers and earlier timesteps, consistently outperform uniform or increasing schedules, leading to improved style fidelity and content preservation.
Forget uniform style injection: strategically decreasing style strength in diffusion models, especially in early timesteps and shallow layers, unlocks a superior style-content tradeoff.
Style transfer with pre-trained diffusion models has advanced rapidly, but a core question remains underexplored: where in the model should style injection be strongest? StyleID, the leading training-free method, uses a single global parameter (gamma) uniformly across all layers and timesteps, which forces a fixed tradeoff between style quality and content preservation. We show this tradeoff is unnecessarily rigid. We systematically explore four dimensions of control: varying style injection strength across decoder layers, across denoising timesteps, and scheduling ControlNet geometric conditioning along both axes. The pattern is consistent everywhere: decreasing schedules, with stronger structural signal injection in shallower layers and earlier timesteps, reliably outperform the reverse. Beyond direction, schedule shape matters: cosine and square-root timestep schedules outperform linear. Most importantly, we find that gamma scheduling and ControlNet conditioning are nearly independent. The resulting combined configurations expand the Pareto frontier, offering superior tradeoffs between style fidelity and content preservation compared to any single baseline setting. Our best balanced configuration achieves ArtFID of 27.036 versus StyleID's 28.801 - a 6.1% relative improvement, with consistent gains across the full style-content tradeoff frontier. Results are validated across 35 configurations totaling over 28,000 stylized images using four complementary metrics. These findings generalize across SD backbones with identical rank ordering. All modifications are training-free, parameter-free, and require only a few lines of scheduling code; code is available at https://github.com/ameyskulkarni/scheduled_style_injection.