Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Random Feature Information Gain (RFIG), a novel approach that leverages random Fourier features to approximate information gain for exploration bonuses in deep reinforcement learning (RL). By grounding RFIG in Bayesian kernel methods, the authors provide theoretical guarantees and error bounds, sidestepping the complexities associated with neural network-based uncertainty estimation. Experimental results show that RFIG not only competes effectively with existing deep exploration methods but also offers clearer theoretical insights, enhancing the understanding of exploration strategies in RL.
RFIG achieves competitive performance in deep RL exploration while providing robust theoretical guarantees, challenging the reliance on complex neural network uncertainty estimations.
Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish. We introduce Random Feature Information Gain (RFIG), grounded in Bayesian kernel methods theory, which uses random Fourier features to approximate information gain and compute exploration bonuses in non-countable spaces. We provide error bounds on information gain approximation and avoid the black-box aspects of neural network-based uncertainty estimation, for optimism-based exploration. We present practical details that make RFIG scalable to deep RL scenarios, enabling smooth integration into standard deep RL algorithms. Experimental evaluation across diverse control and navigation tasks demonstrates that RFIG achieves competitive performance with well-established deep exploration methods while offering superior theoretical interpretation.