Search papers, labs, and topics across Lattice.
The paper explores learning per-domain generalizing policies for planning by training Q-value functions with graph neural networks, offering a more efficient alternative to state-value functions. A key finding is that standard supervised learning of Q-values fails to differentiate between taken and untaken actions, leading to poor performance. To address this, the authors introduce regularization terms that enforce this distinction, resulting in Q-value policies that outperform state-value policies and are competitive with the LAMA-first planner across multiple domains.
Q-value policies, traditionally outperformed by state-value policies in planning, can surpass them with the right regularization, offering a faster alternative for policy evaluation.
Learning per-domain generalizing policies is a key challenge in learning for planning. Standard approaches learn state-value functions represented as graph neural networks using supervised learning on optimal plans generated by a teacher planner. In this work, we advocate for learning Q-value functions instead. Such policies are drastically cheaper to evaluate for a given state, as they need to process only the current state rather than every successor. Surprisingly, vanilla supervised learning of Q-values performs poorly as it does not learn to distinguish between the actions taken and those not taken by the teacher. We address this by using regularization terms that enforce this distinction, resulting in Q-value policies that consistently outperform state-value policies across a range of 10 domains and are competitive with the planner LAMA-first.
