Search papers, labs, and topics across Lattice.
FontFusion introduces a novel conditioning framework for Diffusion Transformer architectures that enhances typography generation by balancing font control and text legibility. The approach employs a hierarchical token representation, position-aware embeddings, and a multi-level token dropping strategy to achieve significant improvements in both computational efficiency and generalization to new fonts. Evaluation results show a 76% relative improvement on decorative fonts and substantial gains in font consistency compared to unconditioned models, all while maintaining compatibility with existing architectures.
Achieving a 76% boost in decorative font generation without sacrificing text legibility, FontFusion redefines typography in diffusion models.
Typography generation in diffusion models faces a persistent trade-off: enabling precise font control typically degrades text legibility, while maintaining readability often sacrifices typographic fidelity. We present FontFusion, a plug-and-play conditioning framework for Diffusion Transformer (DiT) architectures that resolves this dilemma through three core innovations: (1) a hierarchical token representation establishing explicit text-font relationships at multiple granularities, (2) position-aware embeddings creating spatial bindings between typography and image content, and (3) a multi-level token dropping strategy improving both computational efficiency and generalization to unseen fonts. Our systematic evaluation of font embedding spaces reveals that a dual encoder combining DeepFont and DINOv2 outperforms any single encoder for typography tasks. FontFusion demonstrates 76% relative improvement on challenging decorative fonts over single-encoder baselines and font consistency gains exceeding approximately 68-76% over unconditioned models, while integrating into existing DiT architectures without retraining.
