Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
SketchKeyAnime introduces a novel video diffusion framework that synthesizes animations from sparse key-sketch inputs and a single reference RGB image, addressing the limitations of traditional animation methods that require extensive manual input. By employing a dual-branch conditioning mechanism and Sketch Cross Attention, the framework effectively encodes both local geometric constraints and semantic-temporal context, resulting in structurally controllable and appearance-consistent animations. Experimental results demonstrate that SketchKeyAnime significantly outperforms existing baselines in terms of sketch fidelity and temporal coherence, achieving a 31.9% reduction in EDMD and a 9.5% reduction in FVD.
Achieving superior animation synthesis with just a few key sketches and a single reference image, SketchKeyAnime redefines low-cost animation production.
Traditional animation production relies heavily on manual drawing and iterative refinement, particularly for key-pose design, in-betweening, and character coloring. While existing animation and video generation methods have made notable progress, they typically depend on RGB boundary frames, dense frame-wise conditions, or complete sketch sequences, limiting their applicability under low-cost input conditions. We present SketchKeyAnime, a video diffusion framework for generating structurally controllable, appearance-consistent, and temporally coherent animations from sparse key-sketch inputs. Given a single reference RGB image and a few temporally indexed key sketches, SketchKeyAnime introduces a dual-branch conditioning mechanism to encode local geometric constraints alongside semantic-temporal context. It leverages Sketch Cross Attention to fuse reference image and sketch conditions with learnable gating, and incorporates an Adaptive Weighted Loss to strengthen supervision on key-sketch frames and line-art regions. Experimental results on the Aesthetic subset of Sakuga-42M show that our approach consistently outperforms representative animation interpolation and sketch-guided generation baselines. Compared to the best-performing baseline, SketchKeyAnime reduces EDMD by 31.9\% and FVD by 9.5\%, demonstrating superior sketch fidelity and temporal coherence, while achieving the best overall performance across most quantitative metrics. These results validate the proposed framework and highlight its potential for low-cost, highly controllable animation creation.