Search papers, labs, and topics across Lattice.
This paper introduces PWM-ArtGen, a novel Part World Model that addresses the challenge of generating articulated 3D objects from a single image by learning the joint distribution of visual dynamics and kinematic parameters. By coupling action diffusion and image diffusion with independent timesteps, the model effectively utilizes a newly curated dataset of 19.7k part-level image pairs, enabling robust co-training without the need for kinematic annotations. Experimental results show that PWM-ArtGen significantly outperforms existing methods in both standard and zero-shot scenarios, demonstrating its potential for generalization to complex, real-world articulated objects.
PWM-ArtGen achieves remarkable zero-shot generalization for articulated object generation, outperforming traditional methods that struggle with kinematic relationships.
The key challenge in articulated 3D object generation from a single image is accurately predicting the underlying kinematic structure. Existing methods either infer kinematic parameters directly from a static image that lacks dynamic part-level kinematic relationships, or estimate parameters from visual dynamics generated from a single image, which is prone to accumulated errors of two steps. Moreover, the limited scale and diversity of existing annotated datasets further hinder generalization to complex, real-world objects. To overcome these limitations, we propose to learn the joint distribution of visual dynamics and kinematic parameters. Recognizing that articulated objects can be formulated as dynamic systems, we propose a unified Part World Model called PWM-ArtGen. To leverage unannotated data, this model couples action diffusion and image diffusion with independent diffusion timesteps, which enables visual branch co-training. We further curate a photorealistic dataset of 19.7k part-level image pairs without kinematic annotations, to support co-training. Experiments demonstrate that PWM-ArtGen substantially outperforms existing baselines in the resting state and exhibits strong zero-shot generalization to out-of-distribution objects.
