Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study employs the TD3 reinforcement learning algorithm to train autonomous agents for navigation in chaotic double-gyre fluid flows, addressing the challenges posed by partial observability and environmental unpredictability. By evaluating five bio-inspired sensory strategies, the research identifies that agents utilizing localized flow velocity measures and memory achieve superior navigation performance, while explicit global flow parameters hinder effectiveness. The findings reveal a critical trade-off between sensor utility and energy efficiency, providing insights into enhancing bio-inspired robotic navigation in real-world scenarios.
Agents that leverage localized sensory cues can outperform those relying on global flow information, revealing a surprising trade-off in navigation strategies.
Autonomous robotic navigation in nonstationary time-varying fluid flows remains a fundamental challenge due to partial observability and the unpredictability of realistic environments. While classical optimal control frameworks employed in robotics require unrealistic a-priori global flow knowledge, biological systems are able to navigate successfully by exploiting localized sensory cues. In this work we present a reinforcement learning approach using the TD3 algorithm to train autonomous agents to reach arbitrary targets within a parametric, chaotic double-gyre flow. To investigate optimal sensory mechanisms, we evaluate five bio-inspired observation strategies based on relative position, local velocity or local vorticity measures, and short-term memory variants. Additionally, we analyze the impact of providing agents with explicit global flow parameters. Numerical results demonstrate that an agent that is able to sense and remember a set number of flow velocity measures achieves the highest performance. The experiments reveal a trade-off in sensor utility: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity. Incorporating explicit global flow parameters is shown to decrease navigation performance. This behavior suggests that reinforcement learning-based autonomous systems develop more robust and general policies when restricted to implicit flow representations. The presented results offer insights for improving the transition of bio-inspired robotic navigation from simulation to real-world environments.