Search papers, labs, and topics across Lattice.
This paper introduces Sparse Masked Attention Policies (SMAP), a novel approach to improve generalization in reinforcement learning by learning a masking function that operates on attention weights within an attention-based policy network. SMAP aims to address the generalization limitations of abstraction methods by ensuring the information removal process itself generalizes reliably to new states. Experiments on the Procgen benchmark demonstrate that SMAP significantly outperforms standard PPO and other masking approaches in unseen tasks.
By learning to mask attention weights, SMAP enables reinforcement learning agents to generalize far better to unseen environments in Procgen.
In reinforcement learning, abstraction methods that remove unnecessary information from the observation are commonly used to learn policies which generalize better to unseen tasks. However, these methods often overlook a crucial weakness: the function which extracts the reduced-information representation has unknown generalization ability in unseen observations. In this paper, we address this problem by presenting an information removal method which more reliably generalizes to new states. We accomplish this by using a learned masking function which operates on, and is integrated with, the attention weights within an attention-based policy network. We demonstrate that our method significantly improves policy generalization to unseen tasks in the Procgen benchmark compared to standard PPO and masking approaches.
