Search papers, labs, and topics across Lattice.
VARPose introduces a novel approach to densifying 2D sparse poses using Visual Autoregressive Modeling, which enhances the anatomical information for 3D lifting tasks. The method employs a Granularity-agnostic Pose Tokenizer (GPT) to create a unified representation of varying pose densities, allowing for effective decoding of novel granularities. Results show that VARPose not only surpasses existing methods but also significantly improves performance in downstream applications like 3D Pose Estimation and Human Mesh Recovery.
VARPose achieves superior 2D pose densification, leading to enhanced performance in 3D lifting tasks, while generalizing to previously unseen pose granularities.
Visual AutoRegressive Modeling (VAR) has excelled in natural image generation via next-scale prediction, but its use on topology-structured data like human skeletons is still unexplored. VARPose is proposed to adaptively densify 2D sparse poses, thereby enriching the anatomical information available for 3D lifting models. Our core contributions are twofold. First, we introduce a Granularity-agnostic Pose Tokenizer (GPT), which employs a single hybrid codebook and a residual quantization strategy to encode poses of varying densities into a unified, multi-scale discrete representation. Our results demonstrate the strong generalizability of this representation. By decoupling the representation from the projection, we can successfully decode novel pose granularities using a frozen codebook with a retrained decoder. Second, we propose UniSkelar, a unified autoregressive model that treats "joint density" as "scale". UniSkelar learns to predict the token sequence for the next density level in a coarse-to-fine manner, conditioned on the sparsest pose. VARPose not only outperforms state-of-the-art methods and generalizes to unseen granularities, but also confers tangible performance gains on downstream tasks, such as 3D Pose Estimation and Human Mesh Recovery, through 2D pose densification. Our code and model are available at https://github.com/BRL-SYSU/VARPose.git.
