Search papers, labs, and topics across Lattice.
This paper introduces VOID, a novel defense framework designed to combat unauthorized mimicry in Latent Diffusion Models (LDMs) by manipulating their intrinsic stochasticity. Unlike existing methods that rely on deceptive perturbations, VOID employs two innovative strategies: amplifying latent encoding errors to disrupt semantic structure and counteracting target guidance signals to inhibit the model's restoration capabilities. The result is a significant enhancement in security, with the framework achieving a 223% improvement in protection against mimicry attacks while maintaining visual utility.
VOID shatters the semantic structure of images to thwart unauthorized mimicry without sacrificing visual quality, achieving a 223% boost in defense efficacy.
While Latent Diffusion Models (LDMs) have revolutionized visual synthesis, they are increasingly exploited for unauthorized mimicry of individuals. Existing defenses inject deceptive perturbations to steer the generated images toward irrelevant targets. However, this approach hinges on an ungrounded assumption: subtle perturbations can maintain their deceptive efficacy throughout an LDM's extensive generation process. In reality, the model's innate restoration mechanism will remove such perturbations and cause individual identities to re-emerge in the images generated. We propose VOID, a defense framework that overcomes this conundrum by manipulating an LDM's intrinsic stochasticity. VOID perturbs the diffusion pipeline in two novel ways: 1) amplifying the latent encoding errors to shatter an image's semantic structure, and 2) counteracting the target guidance signals to suppress the model's restoration capabilities. This results in a semantic corruption that thwarts any unauthorized mimicry. Notably, the security gain does not come at the price of visual utility, as VOID simultaneously manages to confine perturbations to human-imperceptible regions of protected images. Our comprehensive evaluation of 24 state-of-the-art defenses against 10 mimicry attacks on 5 datasets demonstrates VOID's unprecedented protection power: it increases the average Frechet Inception Distance (FID) from 113 to 365, a 223% improvement over the strongest defense to date.
