Search papers, labs, and topics across Lattice.
This paper introduces Wrench-Augmented Reinforcement Learning (WARL), which enhances exploration in legged robots by incorporating wrench (force and torque) into the action space. The method combines wrench-guided exploration with a success rate-based curriculum to improve learning efficiency across diverse terrains without the need for specific reward adjustments. Experimental results demonstrate that WARL enables robust learning of joint control behaviors, although care must be taken to align wrench-based exploration with the robot's physical capabilities to avoid suboptimal behaviors.
Wrench-based exploration can significantly enhance learning efficiency in legged robots, but misalignment with physical structure may lead to ineffective behaviors.
While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.
