Search papers, labs, and topics across Lattice.
EffectMaker is introduced as a unified reasoning-generation framework for customizable visual effects (VFX) creation, addressing limitations of existing AIGC systems in handling effect-specific data and generalizing to novel VFX. It uses a multimodal LLM for semantic interpretation and reasoning about effect adaptation, combined with a diffusion transformer that leverages in-context learning for fine-grained visual cues. The framework is trained on EffectData, a new synthetic dataset of 130k videos across 3k VFX categories, and demonstrates superior visual quality and effect consistency compared to existing methods.
Skip the costly VFX pipeline: EffectMaker lets you create customized visual effects with a reference video and a multimodal LLM, no per-effect fine-tuning needed.
Visual effects (VFX) are essential for enhancing the expressiveness and creativity of video content, yet producing high-quality effects typically requires expert knowledge and costly production pipelines. Existing AIGC systems face significant challenges in VFX generation due to the scarcity of effect-specific data and the inherent difficulty of modeling supernatural or stylized effects. Moreover, these approaches often require per-effect fine-tuning, which severely limits their scalability and generalization to novel VFX. In this work, we present EffectMaker, a unified reasoning-generation framework that enables reference-based VFX customization. EffectMaker employs a multimodal large language model to interpret high-level effect semantics and reason about how they should adapt to a target subject, while a diffusion transformer leverages in-context learning to capture fine-grained visual cues from reference videos. These two components form a semantic-visual dual-path guidance mechanism that enables accurate, controllable, and effect-consistent synthesis without per-effect fine-tuning. Furthermore, we construct EffectData, the largest high-quality synthetic dataset containing 130k videos across 3k VFX categories, to improve generalization and scalability. Experiments show that EffectMaker achieves superior visual quality and effect consistency over state-of-the-art baselines, offering a scalable and flexible paradigm for customized VFX generation. Project page: https://effectmaker.github.io
