Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to feature selection in reinforcement learning by integrating a non-convex projected minimax concave (PMC) penalty into the least-squares temporal-difference (LSTD) policy evaluation framework, addressing the estimation bias of traditional regularization methods. The authors also extend the theoretical understanding of non-monotone inclusions by establishing new convergence conditions for the forward-reflected-backward splitting (FRBS) method, which applies to a broader class of problems. Experimental results demonstrate that this method significantly outperforms existing feature-selection techniques, particularly in scenarios with a high number of noisy features.
Non-convex regularization in reinforcement learning can dramatically enhance feature selection, outperforming traditional methods in noisy environments.
This work delivers two key contributions: one to efficient feature selection in reinforcement learning (RL), the other to the theory of non-monotone inclusions. On the RL side, the estimation bias inherent in conventional regularization schemes is addressed by augmenting classical least-squares temporal-difference (LSTD) policy evaluation with the sparsity-inducing, non-convex projected minimax concave (PMC) penalty. Because the PMC penalty is weakly convex, the resulting fixed-point problem is no longer monotone; instead, it falls under a broader class of non-monotone inclusions involving the sum of a monotone Lipschitz operator and a hypomonotone operator. On the theory side, novel convergence conditions are developed for the forward-reflected-backward splitting (FRBS) method applied to this broader class of non-monotone inclusion problems. Under mild conditions, Lyapunov stability and the existence of a limit point of the sequence of FRBS iterates are established; alternatively, under the weak Minty variational inequality assumption, exact convergence is guaranteed. Numerical tests on benchmark datasets show that the proposed FRBS iterates, applied to the non-convexly regularized LSTD problem, substantially outperform state-of-the-art feature-selection methods, especially when many noisy features are present.
