Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a method for visualizing latent motion phase structures learned by DRL locomotion policies. The approach extends clustering features beyond state observations to include actions and future states/actions, and introduces a technique to determine the optimal number of clusters by minimizing self-transitions. Applying this method to Ant-v5, HalfCheetah-v5, and Walker2D-v5 environments reveals clearer and more regular phase transition rules compared to methods relying solely on state observations.
Uncovering interpretable phase structures in learned locomotion policies is now possible, revealing insights into how DRL agents internally represent and execute complex movements.
Deep reinforcement learning (DRL) has been shown to achieve high performance on locomotion control tasks in MuJoCo benchmarks such as HalfCheetah, Ant, and Walker2D. However, visualizing the motion structures internally obtained by a trained policy function implemented as a deep neural network remains challenging. It is known from biomechanics and related fields that locomotion control is realized through the repetition of motion phases such as the stance phase and swing phase. In this study, we propose a framework for uncovering latent motion phase structures from trajectories generated by locomotion control policies through interaction with the environment. The proposed method extends the clustering features from state observations alone to augmented features including actions, next states, and next actions, and introduces a method for determining the number of clusters that suppresses self-transitions. Applying the proposed method to three environments -- Ant-v5, HalfCheetah-v5, and Walker2D-v5 -- we successfully identified phase structures with clearer and more regular transition rules than those obtained by the existing method.