Search papers, labs, and topics across Lattice.
This paper introduces the Dreamer-SAC framework, which combines a recurrent state-space world model with an off-policy soft actor-critic algorithm to enhance sample efficiency in autonomous driving. By leveraging both real interactions and short-horizon generated trajectories, the framework addresses the trade-off between data efficiency and model bias, leading to superior performance in driving tasks. Experimental results show that Dreamer-SAC outperforms traditional reinforcement learning baselines while requiring significantly fewer real-world interactions, highlighting an optimal balance between rollout horizon and policy performance.
Achieving superior driving performance with 40% fewer real-world interactions, Dreamer-SAC redefines sample efficiency in autonomous driving.
Sample-efficient reinforcement learning for autonomous driving is often limited by the trade-off between data efficiency and model bias. While world models reduce the reliance on costly environment interactions, policy optimization over learned dynamics remains sensitive to prediction errors. This paper proposes the Dreamer-SAC framework, which integrates a recurrent state-space world model with an off-policy soft actor-critic algorithm trained directly in latent space. The framework uses a combination of real interactions and short-horizon generated trajectories with n-step target estimation and multi-objective supervision. Evaluated in autonomous driving scenarios with objectives encompassing driving efficiency and safety, the proposed framework consistently outperforms representative reinforcement learning baselines, including DreamerV3, SAC, and PPO, while achieving improved performance with substantially fewer real environment interactions. Experiments reveal an inverted-U relationship between rollout horizon and policy performance, where short-horizon latent rollouts achieve the best trade-off between additional training signals and accumulated model bias. Furthermore, n-step target estimation demonstrates more effectiveness over one-step temporal-difference targets in exploiting predicted experience for value learning.
