Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores the operator-theoretic foundations of Q-learning within continuous-time stochastic control, focusing on the regularity and approximation complexity of the Bellman optimality target in a diffusion context. The authors demonstrate that under certain conditions, the Bellman update results in a smoothing effect on state variables while maintaining Lipschitz continuity with respect to action variables, leading to a compact family of Bellman iterates. Additionally, they introduce a tensor-product DeepONet architecture tailored to the mixed regularity of the problem and provide explicit bounds on approximation and resource usage, highlighting a stiffness-complexity trade-off as the time step approaches zero.
Bellman updates in Q-learning can smooth state variables while preserving action variable sensitivity, revealing a new layer of complexity in continuous stochastic control.
We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions. In value-based reinforcement learning, each Q-learning or DQN update is built from a Bellman optimality target; our analysis isolates this target in a diffusion setting and studies its regularity and approximation complexity. Under uniform ellipticity and Hölder-regular coefficients, we show that a Bellman update maps bounded inputs into an anisotropic regularity class, smoothing the state variable while leaving only Lipschitz dependence on the action variable. This yields a compact family of Bellman iterates and motivates a tensor-product DeepONet architecture adapted to the mixed regularity of the problem. We then derive explicit approximation and resource bounds, together with a stiffness--complexity trade-off as the time step $δ\to 0$. The resulting theory makes a direct contribution to Q-learning theory at the level of Bellman target regularity and approximation in continuous stochastic control. At the same time, we do not claim a full convergence theorem for practical sampled Q-learning with exploration, replay, and stochastic gradient updates.