Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Standard continuous reinforcement learning benchmarks rely heavily on forgiving environments where noisy execution can be corrected over time, failing catastrophically on irreversible tasks governed by sharp, ridge-like reward landscapes. To solve this in computational billiards, the authors formulate State-Conditioned Shooting (SCOOT), augmenting advantage-weighted regression with elite-sample policy optimization, a mixture-of-experts architecture, and distance-regularized curriculum learning. The resulting framework achieves ultra-fine physical precision and successfully discovers multiple distinct, high-reward execution strategies for identical table configurations.
Irreversible physical actions create razor-thin reward ridges where conventional continuous RL collapses, a failure mode resolved here by coupling elite-sample filtering with mixture-of-experts policy switching.
Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprecise actions may be corrected with later actions, thus allowing high returns with noisy actions. In contrast, we focus on an under-researched class of high-risk, high-precision motion control problems where actions carry irreversible outcomes, driving sharp peaks and ridges to plague the state-action reward landscape. Using computational pool as a representative example of such problems, we propose and evaluate State-Conditioned Shooting (SCOOT), a novel DRL algorithm that builds on advantage-weighted regression (AWR) with three key modifications: 1) Performing policy optimization only using elite samples, allowing the policy to better latch on to the rare high-reward action samples; 2) Utilizing a mixture-of-experts (MoE) policy, to allow switching between reward landscape modes depending on the state; 3) Adding a distance regularization term and a learning curriculum to encourage exploring diverse strategies before adapting to the most advantageous samples. We showcase our features' performance in learning physically-based billiard shots demonstrating high action precision and discovering multiple shot strategies for a given ball configuration.