Search papers, labs, and topics across Lattice.
This paper introduces an actor-critic framework for solving continuous-time stochastic control problems with jump-diffusion dynamics, using a time-inhomogeneous little q-function and occupation measure to derive a policy gradient representation. The actor is parameterized using conditional normalizing flows to represent expressive, non-Gaussian stochastic policies. The method is validated on linear-quadratic control, Merton portfolio optimization, and a multi-agent portfolio game, demonstrating stable learning and accurate approximation of optimal policies.
Normalizing flows enable stable actor-critic learning for complex stochastic control problems with jumps, opening the door to more realistic financial and economic models.
Continuous-time stochastic control with time-inhomogeneous jump-diffusion dynamics is central in finance and economics, but computing optimal policies is difficult under explicit time dependence, discontinuous shocks, and high dimensionality. We propose an actor-critic framework that serves as a mesh-free solver for entropy-regularized control problems and stochastic games with jumps. The approach is built on a time-inhomogeneous little q-function and an appropriate occupation measure, yielding a policy-gradient representation that accommodates time-dependent drift, volatility, and jump terms. To represent expressive stochastic policies in continuous-action spaces, we parameterize the actor using conditional normalizing flows, enabling flexible non-Gaussian policies while retaining exact likelihood evaluation for entropy regularization and policy optimization. We validate the method on time-inhomogeneous linear-quadratic control, Merton portfolio optimization, and a multi-agent portfolio game, using explicit solutions or high-accuracy benchmarks. Numerical results demonstrate stable learning under jump discontinuities, accurate approximation of optimal stochastic policies, and favorable scaling with respect to dimension and number of agents.
