Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning

by   Gokhan Inalhan, et al.

In this study, reinforcement learning (RL)-based centralized path planning is performed for an unmanned combat aerial vehicle (UCAV) fleet in a human-made hostile environment. The proposed method provides a novel approach in which closing speed and approximate time-to-go terms are used in the reward function to obtain cooperative motion while ensuring no-fly-zones (NFZs) and time-of-arrival constraints. Proximal policy optimization (PPO) algorithm is used in the training phase of the RL agent. System performance is evaluated in two different cases. In case 1, the warfare environment contains only the target area, and simultaneous arrival is desired to obtain the saturated attack effect. In case 2, the warfare environment contains NFZs in addition to the target area and the standard saturated attack and collision avoidance requirements. Particle swarm optimization (PSO)-based cooperative path planning algorithm is implemented as the baseline method, and it is compared with the proposed algorithm in terms of execution time and developed performance metrics. Monte Carlo simulation studies are performed to evaluate the system performance. According to the simulation results, the proposed system is able to generate feasible flight paths in real-time while considering the physical and operational constraints such as acceleration limits, NFZ restrictions, simultaneous arrival, and collision avoidance requirements. In that respect, the approach provides a novel and computationally efficient method for solving the large-scale cooperative path planning for UCAV fleets.


page 1

page 2


Indoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning

In this study, reinforcement learning was applied to learning two-dimens...

Feasible Computationally Efficient Path Planning for UAV Collision Avoidance

This paper presents a robust computationally efficient real-time collisi...

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

The major challenges of collision avoidance for robot navigation in crow...

Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning

This paper introduces a hybrid algorithm of deep reinforcement learning ...

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called ...

Long Short-Term Memory for Spatial Encoding in Multi-Agent Path Planning

Reinforcement learning-based path planning for multi-agent systems of va...

Please sign up or login with your details

Forgot password? Click here to reset