Deep Reinforcement Learning-Based Multirestricted Dynamic-Request Transportation Framework.

Area of Science:

• Robotics and Automation
• Artificial Intelligence
• Logistics and Supply Chain Management

Background:

• Unmanned aerial vehicles (UAVs) are increasingly vital in various sectors, with growing demand for technological advancement.
• Timely delivery of urgent medical supplies during emergencies is critical, but current UAVs face payload and battery limitations.
• Dynamic and uncertain delivery requests pose challenges for efficient UAV-based logistics.

Purpose of the Study:

• To develop a novel framework for optimizing UAV-based urgent medical delivery in dynamic environments.
• To address the limitations of UAVs, including payload capacity and battery life.
• To manage previously unknown delivery requests with specific time constraints.

Main Methods:

• Utilized deep reinforcement learning (DRL) algorithms: Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C).
• Developed a framework to handle dynamically requested packages with unknown source-destination pairs and delivery time intervals.
• Implemented an extended Brute-force (BF) algorithm for comparison, assuming complete prior knowledge of requests and environment.

Main Results:

• Proximal Policy Optimization (PPO) demonstrated superior training performance, exhibiting faster and more stable convergence compared to DQN and A2C.
• The PPO algorithm achieved a success rate comparable to the Brute-force (BF) algorithm in optimizing delivery routes.
• Experimental results validated the effectiveness of the proposed DRL framework in managing uncertain and dynamic delivery scenarios.

Conclusions:

• The proposed DRL framework, particularly using PPO, offers an effective solution for optimizing urgent medical deliveries via UAVs.
• PPO provides a robust and efficient method for handling the complexities of dynamic delivery requests and UAV constraints.
• This research highlights the potential of AI-driven logistics to enhance emergency response capabilities through advanced UAV operations.