Transformer-Based Soft Actor-Critic for UAV Path Planning in Precision Agriculture IoT Networks

Area of Science:

• Robotics and Artificial Intelligence
• Agricultural Technology
• Internet of Things (IoT)

Background:

• Multi-agent path planning for Unmanned Aerial Vehicles (UAVs) faces challenges in efficiency and conflict avoidance, particularly in complex agricultural data collection.
• Existing multi-agent reinforcement learning (MARL) algorithms struggle with high-dimensional state spaces, continuous actions, and inter-agent dependencies.

Purpose of the Study:

• To propose a novel algorithm, Multi-Agent Transformer-based Soft Actor-Critic (MATRS), for safe and efficient collaborative data collection and trajectory optimization by UAVs.
• To address the limitations of current MARL algorithms in handling complex multi-agent coordination tasks.

Main Methods:

• Developed MATRS, utilizing the Centralized Training with Decentralized Execution (CTDE) paradigm.
• Integrated a Transformer encoder with a self-attention mechanism into the critic network to model inter-agent relationships and improve joint action-value function evaluation.

Main Results:

• MATRS demonstrated faster convergence and reduced task completion times compared to baseline algorithms (MADDPG, MATD3, MASAC) in simulations.
• Scalability experiments revealed MATRS learned an effective 'task-space partitioning' strategy for autonomous, conflict-free area coverage by the UAV swarm.

Conclusions:

• Combining attention-based architectures with Soft Actor-Critic learning provides a powerful and scalable solution for multi-UAV coordination in IoT data collection.
• MATRS offers improved performance and autonomous operational strategies for complex agricultural data gathering missions.