Search papers, labs, and topics across Lattice.
This paper introduces SEFS, a novel style-encoder-free conditioning framework for diffusion transformers that separates target geometry from transferable appearance in reference-based diffusion stylization. By utilizing stochastic low-resolution crops of single training images, SEFS effectively preserves local appearance statistics while minimizing unintended scene structure transfer. The method demonstrates improved content consistency and reduced leakage diagnostics on artistic stylization benchmarks, showcasing its efficacy over traditional tuning-based approaches that rely on aligned triplets or auxiliary encoders.
SEFS achieves superior artistic stylization by leveraging low-resolution image crops, enhancing content consistency while avoiding unwanted style transfer artifacts.
Reference-based diffusion stylization requires separating target geometry from transferable appearance. Existing tuning-based methods often rely on aligned content-style-target triplets or auxiliary visual encoders, which increases data cost and can transfer unintended scene structure from the style reference. We propose SEFS (Style-Encoder-Free Stylization), a style-encoder-free conditioning framework for diffusion transformers. SEFS forms style tokens from stochastic low-resolution crops of single training images. This crop bottleneck preserves local appearance statistics such as palette, stroke, texture, and material, while reducing access to global layout cues. Target content is encoded by edge and segmentation cues and fused with the noisy latent through parameter-efficient trainable projections. We add style-to-denoising re-normalization for token-statistic alignment and cross-block skip fusion for spatial detail. SEFS trains on unpaired single images; the frozen diffusion VAE is used only to place image conditions in the latent space. On artistic stylization benchmarks, SEFS improves content consistency and leakage diagnostics while retaining reference-style affinity, and ablations support the crop-resolution, re-normalization, and skip-fusion choices. The code of SEFS will be made publicly available.
