Search papers, labs, and topics across Lattice.
This paper introduces SOMA, a novel approach that infers muscle deformations from surface observations using RGB cameras, addressing the limitations of traditional biomechanical models that fail to capture dynamic muscle behavior. By leveraging a new dataset, SKIM, which provides subject-specific soft-tissue deformations, SOMA enables the generation of anatomically accurate animations without the computational burden of conventional simulations. The results demonstrate that this method offers a scalable and cost-effective solution for creating realistic virtual humans in various applications, from medicine to entertainment.
Recovering muscle deformations from surface observations could revolutionize how we create realistic virtual humans, making high-fidelity animations accessible and efficient.
With the growing demand for realistic virtual humans, parametric body models have become a cornerstone of modern medicine, sports, and entertainment applications. However, most of these models are inherently limited: they only capture the 3D surface of the skin, offering no insight into the complex bio-mechanical structures that generate motion. As more applications expand towards biomechanics, the need for virtual human models that go beyond the skin has become increasingly evident. Traditional soft-tissue simulations, such as FEM, are accurate but non-scalable and too computationally expensive for most common applications. Alternatively, existing biomechanical tools can simulate muscular forces and activations, but do not model changes in external shape, restricting how activations correlate with actual observable anatomy. This motivates a novel inverse research problem: recovering muscle deformations directly from visible surface observations - i.e., from the skin, and thus the pose. In this work, we present SOMA (from Surface Observations to Muscle Anatomy), a person-specific model that infers spatio-temporal muscle behavior from surface signals obtained using RGB cameras, and SKIM, a subject-specific soft-tissue deformation dataset. To the best of our knowledge, this is the first method that attempts to recover muscle deformations from multi-view RGB data. We show how our method provides anatomically grounded animations without the complexity of traditional simulations, leading to a scalable and cost-effective solution. Data and code are available.
