Search papers, labs, and topics across Lattice.
Diffusion Templates is introduced as a unified plugin framework to address the fragmentation of controllable diffusion methods, which are typically developed as isolated systems with incompatible pipelines. The framework decouples base-model inference from controllable capability injection using Template models, a Template cache for standardized interface, and a Template pipeline for loading and merging caches. The framework supports heterogeneous capability carriers like KV-Cache and LoRA under a single abstraction, enabling a diverse model zoo spanning various controllable generation tasks.
Finally, a plugin framework that lets you mix-and-match KV-Cache, LoRA, and other controls to steer diffusion models without being locked into a specific backbone.
Controllable diffusion methods have substantially expanded the practical utility of diffusion models, but they are typically developed as isolated, backbone-specific systems with incompatible training pipelines, parameter formats, and runtime hooks. This fragmentation makes it difficult to reuse infrastructure across tasks, transfer capabilities across backbones, or compose multiple controls within a single generation pipeline. We present Diffusion Templates, a unified and open plugin framework that decouples base-model inference from controllable capability injection. The framework is organized around three components: Template models that map arbitrary task-specific inputs to an intermediate capability representation, a Template cache that functions as a standardized interface for capability injection, and a Template pipeline that loads, merges, and injects one or more Template caches into the base diffusion runtime. Because the interface is defined at the systems level rather than tied to a specific control architecture, heterogeneous capability carriers such as KV-Cache and LoRA can be supported under the same abstraction. Based on this design, we build a diverse model zoo spanning structural control, brightness adjustment, color adjustment, image editing, super-resolution, sharpness enhancement, aesthetic alignment, content reference, local inpainting, and age control. These case studies show that Diffusion Templates can unify a broad range of controllable generation tasks while preserving modularity, composability, and practical extensibility across rapidly evolving diffusion backbones. All resources will be open sourced, including code, models, and datasets.
