Search papers, labs, and topics across Lattice.
This paper addresses the challenge of dynamic scene reconstruction by introducing two approaches for multi-deformation modeling within 3D Gaussian representations, specifically through Mixture of Deformation Experts (MoDE) and Mixture of Experts for Dynamic Gaussian Splatting (MoE-GS). MoDE integrates multiple deformation experts directly into the Gaussian Splatting pipeline via joint optimization, while MoE-GS allows for independent optimization of experts followed by a routing stage for combination. The findings reveal that the choice of integration constraint significantly influences the performance and robustness of dynamic 3D representations in varying motion scenarios.
Multi-deformation modeling can be achieved more effectively by choosing the right integration strategy, impacting the robustness of dynamic scene reconstruction.
Dynamic scene reconstruction remains challenging due to the heterogeneous and spatially varying nature of real-world motion. Although recent 3D Gaussian Splatting methods have introduced diverse deformation formulations for dynamic novel view synthesis, each method typically relies on a single deformation model within its representation, which limits robustness across diverse dynamic scenarios. In this work, we study a fundamental problem-multi-deformation modeling for dynamic 3D Gaussian representations-under two distinct integration constraints that differ in when and how multiple deformation experts interact during training. From a Mixture-of-Experts (MoE) perspective, we view multi-deformation modeling as the problem of combining multiple specialized deformation models within a unified 3D representation. We first introduce Mixture of Deformation Experts (MoDE), which integrates multiple deformation experts directly into the deformable Gaussian Splatting pipeline through joint optimization. In MoDE, experts operate on a shared canonical Gaussian representation, enabling multi-deformation modeling without introducing additional training stages or modifying the original optimization schedule. In contrast, we further present Mixture of Experts for Dynamic Gaussian Splatting (MoE-GS) under a different integration constraint, where deformation experts are optimized independently and combined through a separate routing stage. As a result, expert interaction occurs over non-canonical Gaussian representations after individual optimization. Together, these two approaches provide alternative strategies for multi-deformation modeling, clarifying how integration constraints shape the design and behavior of deformation experts in dynamic 3D Gaussian representations. Our code is available at: https://github.com/cvsp-lab/MoE-GS-studio.
