Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
The authors develop Representation Injection (REPI), a training framework for diffusion transformers that inverts the standard representation alignment paradigm by actively injecting pretrained visual encoder features directly into the transformer's denoising layers. By treating these injected features as a temporary scaffold that is progressively decayed, the model fully internalizes the external semantic representations without incurring inference overhead. REPI consistently outperforms REPA across backbones, and their combination matches a 7M-step vanilla SiT baseline in just 160K steps, delivering a >43.5× pretraining speedup.
Treating pretrained vision embeddings as temporary scaffolding to inject and internalize—rather than passive alignment targets—slashes diffusion transformer training budgets by over 43×.
Recent representation alignment (REPA) methods accelerate diffusion transformer training by aligning projections of the transformer's hidden states with representations from pretrained visual encoders. In this work, we explore a reverse and complementary direction to REPA: rather than projecting diffusion representations into the encoder's space, we inject encoder representations into the diffusion transformer, allowing them to actively participate in the denoising process. To this end, we introduce \textit{REPresentation Injection} (REPI), a training framework based on a scaffold-to-internalization strategy, in which projected encoder representations initially serve as a temporary scaffold and are then progressively internalized by the diffusion transformer. REPI outperforms REPA across a wide range of backbones and is highly complementary to it: combining the two yields substantial gains over either alone. Notably, with only 160K training steps, REPI + REPA matches vanilla SiT trained for 7M steps, a speedup of over $43.5\times$. Code will be available at https://jeneveuxpas.github.io/REPI