Search papers, labs, and topics across Lattice.
FreeShadow introduces a novel training-free approach to shadow removal by leveraging pretrained diffusion models, addressing the limitations of existing methods that struggle with generalization and artifacts. The method employs illumination transfer attention (ITA) to re-weight self-attention maps, effectively transferring illumination cues from non-shadow to shadow regions while preserving content fidelity. Extensive experiments reveal that FreeShadow achieves strong generalization and produces realistic shadow-free images without the need for training or optimization.
FreeShadow achieves realistic shadow removal without any training, leveraging diffusion models to generalize effectively across diverse scenarios.
Existing supervised and unsupervised shadow removal methods often suffer from limited generalization due to the insufficient diversity of available training datasets, while zero-shot methods tend to produce artifacts and require time-consuming test-time optimization. To address these issues, we propose FreeShadow, a training-free shadow removal method built upon pretrained diffusion models, which exploits diffusion priors for shadow removal without any training or optimization. For illumination recovery, we propose an illumination transfer attention (ITA), which re-weights the self-attention maps in diffusion model to transfer illumination cues from non-shadow to shadow regions. For content preservation, we analyze the effects of illumination variations on self-attention maps and latent high-frequency features in diffusion model, and selectively preserve illumination-invariant components to maintain content fidelity while suppressing residual shadows. We further propose local texture-preserving relighting (LTPR) to mitigate local texture misalignment caused by VAE compression. Extensive experiments demonstrate that our method achieves strong generalization and produces realistic shadow-free images.
