Search papers, labs, and topics across Lattice.
This paper introduces MuNet, a novel neural network architecture that jointly tackles 3D human mesh recovery and 3D clothed human reconstruction from single images by leveraging their mutual dependencies. MuNet uses 2-manifold graphs as a unified representation and an end-to-end graph convolutional network to progressively deform an initial graph into a 3D human mesh and refine it into a detailed 3D clothed human model. A mutualistic mechanism enables reciprocal interaction between the two tasks during training, leading to state-of-the-art performance on six benchmark datasets.
Jointly optimizing 3D human mesh recovery and clothed human reconstruction yields state-of-the-art results, proving that tackling these tasks together is better than addressing them in isolation.
3D human mesh recovery and 3D clothed human reconstruction are inherently related, yet they have long been studied in isolation, thereby overlooking the potential gains of joint optimization. To overcome this limitation, we propose to address these two tasks within a unified framework, which allows their mutual dependencies to be effectively exploited. Building on this idea, we propose MuNet, a mutualistic network for joint 3D human mesh recovery and 3D clothed human reconstruction from single images. First, we adopt 2-manifold graphs as a unified representation for all 3D models, enabling consistent modeling across 3D human mesh recovery and clothed human reconstruction. Second, we design an end-to-end graph convolutional network that progressively deforms an initial graph into a 3D human mesh and refines it into a detailed 3D clothed human model. Third, we introduce a mutualistic mechanism that allows reciprocal interaction between the two tasks {during training}, where 3D human mesh recovery provides guidance for 3D clothed human reconstruction, and reconstruction feedback refines the 3D human mesh recovery. We extensively evaluate MuNet on six benchmark datasets for 3D human mesh recovery and 3D clothed human reconstruction, including Human3.6M, 3DPW, MPI-INF-3DHP, THuman2.0, CAPE, and RenderPeople. Experimental results demonstrate that MuNet achieves state-of-the-art performance on both tasks across all datasets. The code of MuNet is released for research purposes at https://github.com/starVisionTeam/MuNet.
