Search papers, labs, and topics across Lattice.
This paper addresses the challenges of deploying vision-based convolutional neural networks (CNNs) on ultra-low-power (ULP) multicore processors in autonomous nano-drones, specifically the Crazyflie 2.1. By automating the end-to-end optimization process, the authors achieve a twofold reduction in memory footprint and a 1.6x speedup in inference time while maintaining prediction accuracy. The optimized system demonstrates enhanced navigation capabilities, including improved obstacle avoidance and lane following, while consuming less than 1.6% of the drone's power budget.
Achieving a 1.6x speedup and 50% reduction in memory usage for CNNs on nano-drones could redefine their operational efficiency and application scope.
The evolution of energy-efficient ultra-low-power (ULP) parallel processors and the diffusion of convolutional neural networks (CNNs) are fueling the advent of autonomous driving nano-sized unmanned aerial vehicles (UAVs). These sub-10cm robotic platforms are envisioned as next-generation ubiquitous smart-sensors and unobtrusive robotic-helpers. However, the limited computational/memory resources available aboard nano-UAVs introduce the challenge of minimizing and optimizing vision-based CNNs – which to date require error-prone, labor-intensive iterative development flows. This work explores methodologies and software tools to streamline and automate all the deployment of vision-based CNN navigation on a ULP multicore system-on-chip acting as a mission computer on a Crazyflie 2.1 nano-UAV. We focus on the deployment of PULP-Dronet (Palossi <italic>et al.</italic>, 2019), a state-of-the-art CNN for autonomous navigation of nano-UAVs, from the initial training to the final closed-loop evaluation. Compared to the original hand-crafted CNN, our results show a <inline-formula> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> reduction of memory footprint and a speedup of <inline-formula> <tex-math notation="LaTeX">$1.6\times $ </tex-math></inline-formula> in inference time while guaranteeing the same prediction accuracy and significantly improving the behavior in the field, achieving: <italic>i</italic>) obstacle avoidance with a peak braking-speed of 1.65m/s and improving the speed/braking-space ratio of the baseline, <italic>ii</italic>) free flight in a familiar environment up to 1.96m/s (0.5m/s for the baseline), and <italic>iii</italic>) lane following on a path featuring a 90deg turn – all while using for computation less than 1.6% of the drone’s power budget. To foster new applications and future research, we open-source all the software design in a ready-to-run project compatible with the Crazyflie 2.1.
ETH