Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the intersection of privacy and reinforcement learning by analyzing the Randomized Least Squares Value Iteration (RLSVI) algorithm in an episodic setting. The authors demonstrate that the noise introduced for randomized exploration in RLSVI inherently provides joint differential privacy, characterized by a specific privacy bound dependent on the number of states, actions, episode length, and episodes. This finding is significant as it establishes a framework for ensuring user privacy in sensitive applications of RL without compromising algorithmic performance.
RLSVI not only optimizes exploration but also guarantees joint differential privacy, making it a game-changer for sensitive RL applications.
As reinforcement learning (RL) increasingly applies to sensitive domains, such as health care and recommendation systems, privacy-preserving techniques have become essential to protect users' sensitive information. We investigate privacy-preserving RL under an episodic setting, focusing on algorithms based on randomized exploration, such as Randomized Least Squares Value Iteration (RLSVI). The overall goal is to study how randomized exploration interacts with the injected noise required by privacy mechanisms. In this work, we show a new privacy analysis that characterizes how the noise in RLSVI set for exploration simultaneously provides privacy protection. Specifically, we prove that RLSVI is $(\varepsilon(δ),δ)$-joint differentially private in tabular MDP as is with $\varepsilon(δ) = \frac{2AK}{H^2\log(2HSA)} + 2\sqrt{\frac{2AK\log(1/δ)}{H^2\log(2HSA)}}$, where $S$ and $A$ are the number of states and actions respectively, $H$ is the length of an episode and $K$ is the number of episodes.