Search papers, labs, and topics across Lattice.
This paper introduces a novel controllable diffusion framework tailored for linear attention backbones, addressing limitations of existing methods like ControlNet and OminiControl when applied to such architectures. The core innovation is a unified gated conditioning module operating in a dual-path pipeline, enabling effective integration of multi-type conditional inputs (both spatially aligned and non-aligned). Experiments across multiple tasks demonstrate state-of-the-art controllable generation performance on linear-attention models, outperforming existing methods in fidelity and controllability.
Linear attention models can now achieve SOTA controllable generation performance, thanks to a new gated conditioning module that overcomes the limitations of ControlNet-style approaches.
Recent advances in diffusion-based controllable visual generation have led to remarkable improvements in image quality. However, these powerful models are typically deployed on cloud servers due to their large computational demands, raising serious concerns about user data privacy. To enable secure and efficient on-device generation, we explore in this paper controllable diffusion models built upon linear attention architectures, which offer superior scalability and efficiency, even on edge devices. Yet, our experiments reveal that existing controllable generation frameworks, such as ControlNet and OminiControl, either lack the flexibility to support multiple heterogeneous condition types or suffer from slow convergence on such linear-attention models. To address these limitations, we propose a novel controllable diffusion framework tailored for linear attention backbones like SANA. The core of our method lies in a unified gated conditioning module working in a dual-path pipeline, which effectively integrates multi-type conditional inputs, such as spatially aligned and non-aligned cues. Extensive experiments on multiple tasks and benchmarks demonstrate that our approach achieves state-of-the-art controllable generation performance based on linear-attention models, surpassing existing methods in terms of fidelity and controllability.
