Search papers, labs, and topics across Lattice.
This paper addresses the challenge of erasing broad, multi-faceted concepts (e.g., "sexual," "violent") in text-to-image diffusion models, where existing methods struggle. The authors propose a "prototype-guided concept erasure" method that identifies latent embeddings encoding the target concept, clusters them to derive concept prototypes, and uses these prototypes as negative conditioning signals during inference. Experiments demonstrate that this approach achieves more reliable removal of broad concepts while maintaining image quality.
Reliably erase broad concepts like "sexual" or "violent" from diffusion models by using learned concept prototypes as negative guidance, outperforming existing methods.
Concept erasure is extensively utilized in image generation to prevent text-to-image models from generating undesired content. Existing methods can effectively erase narrow concepts that are specific and concrete, such as distinct intellectual properties (e.g. Pikachu) or recognizable characters (e.g. Elon Musk). However, their performance degrades on broad concepts such as ``sexual'' or ``violent'', whose wide scope and multi-faceted nature make them difficult to erase reliably. To overcome this limitation, we exploit the model's intrinsic embedding geometry to identify latent embeddings that encode a given concept. By clustering these embeddings, we derive a set of concept prototypes that summarize the model's internal representations of the concept, and employ them as negative conditioning signals during inference to achieve precise and reliable erasure. Extensive experiments across multiple benchmarks show that our approach achieves substantially more reliable removal of broad concepts while preserving overall image quality, marking a step towards safer and more controllable image generation.
