Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Wave2Body, a novel framework for human pose estimation using mmWave radar that decouples the learning of radar perception and body structure through a self-supervised mmWave tokenizer and a pretrained compositional body tokenizer. By addressing the limitations of existing methods that rely on paired radar-pose data, Wave2Body enhances cross-domain generalization while significantly reducing computational costs for both training and inference. Experimental results on the M4Human and mmBody datasets demonstrate superior performance compared to traditional approaches, highlighting the effectiveness of the proposed method in overcoming the challenges of sparse radar observations.
Wave2Body achieves remarkable cross-domain generalization in mmWave human pose estimation while slashing computational costs.
Millimeter-wave (mmWave) radar enables privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry-dependent observations is therefore under-constrained. Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their alignment. This coupling can encourage dataset-specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar-to-body token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross-domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.