Search papers, labs, and topics across Lattice.
The paper introduces SAM 3D Animal, a promptable framework for reconstructing multiple 3D animal instances from single images, addressing limitations of existing single-animal focused methods. They leverage the SMAL+ parametric model and introduce the Herd3D dataset (5K images) to train the model with diverse species, interactions, and occlusions. Experiments on multiple datasets demonstrate state-of-the-art performance compared to both model-based and model-free approaches, showcasing the effectiveness of prompt-driven multi-animal 3D reconstruction.
Reconstructing 3D animals in the wild just got a whole lot easier, even in crowded and occluded scenes, thanks to a new promptable framework.
3D animal reconstruction in the wild remains challenging due to large species variation, frequent occlusions, and the prevalence of multi-animal scenes, while existing methods predominantly focus on single-animal settings. We present SAM 3D Animal, the first promptable framework for multi-animal 3D reconstruction from a single image. Built on the SMAL+ parametric animal model, our method jointly reconstructs multiple instances and supports flexible prompts in the form of keypoints and masks which enable more reliable disambiguation in crowded and occluded scenes. To train such a model, we further introduce Herd3D, a multi-animal 3D dataset containing over 5K images, designed to increase diversity in species, interactions, and occlusion patterns. Experiments on the Animal3D, APTv2, and Animal Kingdom datasets show that our framework achieves state-of-the-art results over both existing model-based and model-free methods, demonstrating a scalable and effective solution for prompt-driven animal 3D reconstruction in the wild.
Tsinghua AI