Meituan UAV Challenge

An open-world embodied navigation and delivery system that turns natural-language requests into safe, precise UAV missions.

Overview

This project explores a more flexible form of aerial delivery: instead of flying only between fixed stations, a UAV can interpret an open-ended request, search an unfamiliar environment, identify a safe drop-off point, and navigate there autonomously. The system connects a vision-language model (VLM) with onboard perception, mapping, planning, and control to turn instructions such as “deliver the package beside the person in a dark-green shirt” into a complete physical mission.

Developed for the creativity track of the Meituan 3rd Low-Altitude Economy Intelligent Flight Management Challenge, the project received the Excellent Creativity Award (Second Place) and was recognized during IROS 2025. The competition was jointly organized by Meituan Academy of Robotics Shenzhen and Tsinghua Shenzhen International Graduate School.

Technical approach

Active semantic exploration
Compresses long-horizon observations into multi-view memory keyframes, asks the VLM to score unexplored regions, and combines semantic value with geometric travel cost for global planning.
Language-to-coordinate grounding
Uses an AirSim data engine and reinforcement fine-tuning to map an open-ended delivery request to a precise image coordinate, followed by 3D projection and geometric safety checks.
Visual-feature-aware planning
Maintains an incremental visual feature map and plans multilevel trajectories that balance localization robustness, obstacle avoidance, and navigation efficiency when GNSS or prior maps are unavailable.
ROS sim-to-real stack
Connects Python-based semantic reasoning with C++ planning and low-level control through consistent ROS interfaces, allowing the same closed loop to run in AirSim and on the custom quadrotor.
A 3D semantic map with explored regions, candidate viewpoints, and planned UAV trajectories
Semantic-geometric planning. The UAV maintains a 3D representation of the environment and repeatedly replans toward semantically promising regions while preserving collision-free, dynamically feasible motion.

Validation

85.5%success rate across more than 100 simulated navigation tasks
89.0%oracle success rate across eight outdoor simulation environments
4.2 maverage final distance to the requested target
134 saverage end-to-end flight time
Four AirSim environments and a delivery drone flying through a coastal scene
Large-scale simulation. More than 100 tasks were evaluated across eight outdoor scenes spanning urban blocks, residential areas, parks, forests, beaches, and waterfront settlements.
A custom quadrotor taking off, exploring a park, and reaching a user carrying a suspended package
Outdoor deployment. Five field missions around Shenzhen's Tanglang Mountain area validated the full sequence from takeoff and VLM-guided exploration to arrival at a user-specified delivery location.

Platform

The 1.7 kg custom quadrotor combines an Intel NUC, an OAK multi-camera perception system, a Livox MID-360 LiDAR, and PX4 flight control. The real-world study covered an outdoor area of approximately 5 km² and used natural-language delivery requests paired with only a coarse GPS prior, requiring the UAV to complete the final semantic search and approach autonomously.

Competition & event