Search papers, labs, and topics across Lattice.
The paper introduces MultiAnimate, a diffusion-transformer-based framework for pose-guided multi-character image animation that addresses identity confusion and implausible occlusions common in naive extensions of single-character animation methods. MultiAnimate incorporates an Identifier Assigner and Identifier Adapter to capture per-person positional cues and inter-person spatial relationships using a mask-driven approach. Trained on a two-character dataset, the model generalizes to animating scenes with more characters while remaining compatible with single-character animation, achieving state-of-the-art performance.
An animation framework trained on just two characters can generalize to animating scenes with many more, without sacrificing single-character performance.
Pose-guided human image animation aims to synthesize realistic videos of a reference character driven by a sequence of poses. While diffusion-based methods have achieved remarkable success, most existing approaches are limited to single-character animation. We observe that naively extending these methods to multi-character scenarios often leads to identity confusion and implausible occlusions between characters. To address these challenges, in this paper, we propose an extensible multi-character image animation framework built upon modern Diffusion Transformers (DiTs) for video generation. At its core, our framework introduces two novel components-Identifier Assigner and Identifier Adapter - which collaboratively capture per-person positional cues and inter-person spatial relationships. This mask-driven scheme, along with a scalable training strategy, not only enhances flexibility but also enables generalization to scenarios with more characters than those seen during training. Remarkably, trained on only a two-character dataset, our model generalizes to multi-character animation while maintaining compatibility with single-character cases. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in multi-character image animation, surpassing existing diffusion-based baselines.
