Search papers, labs, and topics across Lattice.
This paper introduces a novel method for concept removal in advanced image generative models, specifically targeting diffusion and autoregressive architectures like SD3.5, Flux, and Infinity. By replacing the internal bottleneck layer with a transcoder that structures activation features, the method allows for selective disabling of undesirable concepts while preserving overall image quality. Empirical results demonstrate that this approach achieves state-of-the-art performance in concept removal, maintains visual fidelity, and offers robustness against adversarial prompts, making it a significant advancement in the field.
Efficiently removing undesirable concepts from image generations without sacrificing quality could redefine how we manage biases in generative models.
Image generative models are trained on massive, largely uncurated internet-scale datasets that contain undesirable visual concepts. Efficiently removing such concepts from the model generations without degrading the quality of output images remains challenging. We introduce a novel concept removal method for frontier diffusion and image autoregressive models, such as SD3.5, Flux, and Infinity. Our intervention replaces the internal bottleneck layer present in all these modern models with a transcoder that is trained to replicate the original layer while structuring it into distinct activation features. This in-place substitution creates an integrated filter through which concept-specific signals can be selectively disabled while preserving the rest of the model's behavior. Since the intervention modifies the model backbone rather than attaching an external component, it remains persistent under white-box access. Empirically, the approach achieves state-of-the-art concept removal performance across modern diffusion and autoregressive models, maintains visual generation quality, provides robustness against adversarial prompts, and supports sequential removal of diverse concepts. This positions our method as a practical approach for concept removal in frontier image generative models.
