Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Pareto Q-Learning with Reward Machines (PQLRM), a novel multi-objective reinforcement learning algorithm that leverages reward machines to define complex reward structures. By integrating Pareto Q-Learning with enhancements from Q-Learning with Reward Machines, PQLRM efficiently approximates the Pareto front while maintaining sample efficiency even in non-Markovian contexts. Experimental results demonstrate that PQLRM converges more rapidly than traditional PQL methods and can synthesize Pareto-optimal policies that are unattainable through standard QRM approaches.
PQLRM achieves faster convergence and superior policy synthesis in multi-objective reinforcement learning, outperforming traditional methods in complex reward scenarios.
We present Pareto Q-Learning with Reward Machines (PQLRM), a multi-objective reinforcement learning algorithm for tasks whose reward structure is specified by a set of reward machines (RMs). PQLRM combines Pareto Q-Learning (PQL), which maintains sets of vector-valued Q-estimates to approximate the Pareto front, with enhancements from Q-Learning with Reward Machines (QRM), which exploits the factored automaton structure of the reward signal. This yields a multi-policy algorithm that remains sample-efficient under non-Markovian, RM-encoded rewards. Experimental trials show that PQLRM converges faster than a naive PQL baseline applied to the cross-product MDP and can synthesize Pareto-optimal policies that QRM cannot.