Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study explores a physics-aware end-to-end deep reinforcement learning (DRL) framework for quadcopter control that directly manipulates low-level body inputs while accounting for actuator dynamics. By integrating a high-fidelity Simulink simulator with a 12-state rigid-body model and incorporating first-order actuator dynamics, the authors evaluate four DRL algorithms—DDPG, TD3, PPO, and SAC—across two control scenarios. The findings reveal that SAC and TD3 outperform the others in terms of stability and exploration efficiency, underscoring the importance of accurately modeling actuator lags and aerodynamic effects for effective quadcopter control.
SAC and TD3 not only excel in stability but also redefine exploration efficiency in quadcopter control, challenging the conventional wisdom around DRL algorithm performance.
Unmanned aerial vehicles (UAVs), particularly quadcopters, present unique challenges for autonomous control due to their underactuated dynamics: only four available control inputs must govern six degrees of freedom. This paper investigates a physics-aware, end-to-end deep reinforcement learning (DRL) approach that acts directly on low-level body inputs, total thrust and body torques $(T, \tau_x, \tau_y, \tau_z)$, and closes the loop through a high-fidelity Simulink environment. Our simulator integrates a 12-state rigid-body model (MATLAB Level-2 S-Function) with (i) an Action2RPM allocation based on the Moore-Penrose pseudo-inverse of a coefficient matrix derived from thrust and drag terms, and (ii) first-order actuator dynamics for each motor (time constant $T_m = 0.076$ s), including rotor gyroscopic coupling. A shaped reward balances goal-reaching and stability using an exponential position well, attitude penalties, and quadratic velocity costs. Four DRL algorithms, DDPG, TD3, PPO, and SAC, are evaluated in two stages: (S1) thrust-only hover and (S2) hover with pitch torque and a translated goal. Results show that SAC and TD3 achieve superior stability and exploration efficiency, while PPO is less sample-efficient. The study highlights the significance of modeling actuator lags and aerodynamic moments for stable low-level control and provides a reproducible benchmark for quadcopter DRL.