Search papers, labs, and topics across Lattice.
This paper extends Direct Advantage Estimation (DAE) to partially observable environments, addressing its limitations in applicability and computational overhead. By introducing discrete latent dynamics models, the authors significantly reduce the complexity of modeling transition probabilities, enabling DAE to operate efficiently in high-dimensional settings. Evaluations on the Arcade Learning Environment demonstrate that the modified DAE maintains high sample efficiency while effectively scaling with function approximator capacity.
Scaling Direct Advantage Estimation to partially observable environments could revolutionize sample efficiency in deep reinforcement learning.
Direct Advantage Estimation (DAE) has been shown to improve the sample efficiency of deep reinforcement learning algorithms. However, its reliance on full environment observability limits its applicability in realistic settings, and its requirement to model transition probabilities incurs substantial computational overhead for high-dimensional observations. In the present work, we address both limitations. First, we extend the theoretical framework of DAE to partially observable domains with minimal modifications. Second, we reduce its computational complexity by introducing discrete latent dynamics models that efficiently approximate transition probabilities. We evaluate our approach on the Arcade Learning Environment and find that DAE scales effectively with function approximator capacity while retaining high sample efficiency.
