Search papers, labs, and topics across Lattice.
This paper introduces an LLM-guided distributed model predictive control (MPC) framework for decentralized UAV formations, enabling autonomous swarm control from natural language inputs. A fine-tuned Phi-2 LLM interprets mission objectives into structured plans, which are then executed by individual UAVs using local MPC controllers for trajectory optimization, collision avoidance, and formation maintenance. High-fidelity simulations using ROTORS and Unreal Engine demonstrate the framework's effectiveness in achieving high mission success rates and adaptability to mid-mission updates.
Forget complex interfaces: control drone swarms with natural language, thanks to an LLM-powered system that translates your commands into coordinated flight plans.
Real-time autonomous control of decentralized drone swarms in dynamic and cluttered environments remains a significant challenge. This paper presents a natural language-driven framework that integrates a fine-tuned large language model (LLM) with distributed model predictive control (MPC) to enable scalable and responsive UAV swarm autonomy. The system architecture comprises a ground control unit, an intelligent mission planning agent, and a decentralized swarm of drones. Mission objectives and target coordinates supplied by external sources (e.g., satellites, command center or airborne platforms), are processed by fine-tuned Phi-2 LLM trained on over 200,000 command variations. The LLM interprets these natural language inputs into structured mission plans, including drone assignments, formations, and operational modes (e.g., swarm-based, multi-target, or single-agent deployments). These plans are dispatched via the Agent mission allocator to the UAVs, each of which leverages a local MPC controller to execute its assigned task. The controllers dynamically optimize flight trajectories while ensuring collision avoidance, formation maintenance, and seamless role transitions. The framework is validated in a high-fidelity simulation environment that combines the ROTORS quadrotor dynamics simulator with Unreal Engine’s photorealistic and depth-aware rendering, facilitating vision-based navigation in cluttered environments. Experimental results demonstrate high mission success rates, accurate formation tracking, and robust adaptability to mid-mission updates, affirming the potential of combining LLM-driven intent parsing with decentralized MPC for intuitive, safe, and scalable swarm control. Future work will focus on extending this framework to physical UAV platforms for real-world deployment.
