Search papers, labs, and topics across Lattice.
This paper introduces ThorArena, a benchmark designed to evaluate humanoid robots' physical interaction capabilities by incorporating synchronized motion and force measurements from human demonstrations. The authors highlight the limitations of existing benchmarks that focus solely on kinematic motion, which fail to account for the impact of external forces on tracking accuracy and control robustness. Through the development of force-aware evaluation metrics and a unified benchmark protocol, the study reveals significant performance discrepancies in humanoid control policies that traditional evaluations overlook, underscoring the importance of force awareness in robotics research.
Force-aware evaluation metrics reveal hidden performance gaps in humanoid robot control that conventional benchmarks miss.
Humanoid robots are increasingly expected to perform contact-rich tasks that require not only accurate whole-body motion but also robust physical interaction with surrounding objects and humans. Although recent advances in humanoid motion imitation and whole-body control have achieved remarkable tracking performance, existing datasets and benchmarks primarily focus on kinematic motion while largely overlooking synchronized interaction forces. As a result, current evaluations fail to capture how external interaction forces affect tracking accuracy, stability, and control robustness. In this paper, we present ThorArena, a benchmark for evaluating force-aware humanoid interaction based on human demonstrations with synchronized motion and force measurements. We collect a real-world interaction dataset that simultaneously captures whole-body human motion and forces exerted by both hands across six representative physical interaction tasks. Based on these demonstrations, we propose force-aware evaluation metrics that jointly assess whole-body tracking accuracy, robustness under different force levels, control effort, and episode survival through the Force-Aware Tracking Score (FATS) and complementary diagnostic metrics. We further establish a unified benchmark protocol that replays recorded interaction forces in simulation and provides a standardized evaluation interface for different humanoid control policies. Experiments on representative whole-body control policies demonstrate that force-aware evaluation reveals substantial performance differences that remain largely hidden under conventional no-force evaluation. ThorArena provides a practical and reproducible framework for studying force-aware humanoid interaction and offers a new benchmark for evaluating contact-rich humanoid behaviors.
