Search papers, labs, and topics across Lattice.
This paper introduces a Multi-Action Tangled Program Graph (MATPG) algorithm, extending the Tangled Program Graph (TPG) approach to continuous control multi-task reinforcement learning (MTRL) environments. MATPG aggregates MAPLE agents and uses a control flow to activate them, adapting genetic programming for continuous action spaces. The authors demonstrate MATPG's effectiveness on a novel MuJoCo Half Cheetah benchmark with multiple obstacles, showing it outperforms other methods when combined with lexicase selection and offering interpretable decision flows.
Evolving interpretable control policies for multi-task robots is now possible: MATPG leverages genetic programming to create a single agent that masters multiple continuous control tasks.
Over the past few decades, machine learning has been widely used to learn complex tasks. Reinforcement Learning (RL), inspired by human behavior, is a great example, as it involves developing specific behaviours for specific tasks. To further challenge algorithms, Multi-Task RL (MTRL) environments have been introduced, requiring a single model to learn multiple behaviors. The Tangled Program Graph (TPG) algorithm is a Genetic Programming (GP) algorithm designed for discrete MTRL environments. Recently, the MAPLE algorithm has been proposed, as another GP algorithm that achieves high results in single task continuous RL environments. A variation of the TPG is proposed alongside MAPLE, named Multi-Action TPG (MATPG) that aggregates MAPLE agents, and creates a control flow to activate them. Initially tested on single task RL environments only, MATPG achieved similar results to MAPLE. In this work, we present a new benchmark based on the MuJoCo Half Cheetah from Gymnasium. This benchmark features five distinct obstacles that are randomly positioned in front of the agent, each of which demands a unique behavior. This benchmark serves as a use case for MATPG, to prove its ability as a GP solution for continuous MTRL environments. Our experiments demonstrate its superiority in this multi-task use case when combined with lexicase selection. Furthermore, we examine the interpretability of the evolved graph, revealing that the decision flow of the model is fully interpretable.
