Search papers, labs, and topics across Lattice.
GenEraser addresses the challenge of removing objects and their associated effects from videos in out-of-domain scenarios by leveraging balanced text-mask guidance and a decoupled locator-preserver architecture. The framework introduces a Multi-Conditional Mixture-of-Experts (MC-MoE) with Bipartite Text guidance to improve effect identification and a Learnable Deep ``CFG''Fusion mechanism (LD-CFG) to balance mask and textual conditions. By decoupling the architecture into Locator and Preserver experts, GenEraser achieves state-of-the-art performance on benchmarks like ROSE and VOR-Eval, demonstrating robust generalization.
Removing objects from video just got a whole lot cleaner: GenEraser doesn't just erase the object, it intelligently removes associated effects like shadows and reflections, setting a new bar for realistic video editing.
Video object removal frequently struggles to simultaneously eliminate target objects and their associated physical effects (e.g., smoke, reflections, light, and ripples) in out-of-domain scenarios due to complex spatiotemporal ambiguities. While existing methods primarily rely on spatial masks, they often fail to capture weakly correlated effects, and the potential of explicit textual guidance remains underexplored. Furthermore, a fundamental optimization conflict exists in removal models between high-level semantic generalization and precise pixel-level background preservation. To address these challenges, we propose GenEraser, a novel framework for generalized and high-fidelity video object and effect removal. First, we introduce a Multi-Conditional Mixture-of-Experts (MC-MoE) paired with Bipartite Text guidance to fully exploit the multimodal priors of Diffusion Transformers, significantly enhancing the identification of complex effects. Second, a Learnable Deep ``CFG''Fusion mechanism (LD-CFG) is developed to adaptively balance the relative dominance of mask and textual conditions across diverse scenarios. Finally, we propose a Decoupled Expert Architecture, comprising a Locator and a Preserver, to mitigate the inherent trade-off between semantic generalization and pixel alignment. Extensive experiments demonstrate that our GenEraser surpasses recent state-of-the-art approaches, achieving significant quantitative improvements (e.g., $2.16$ dB and $1.44$ dB on the ROSE Benchmark and VOR-Eval, respectively) while maintaining exceptionally robust generalization in open-world scenarios. https://cyqii.github.io/GenEraser.github.io/
Tsinghua AI