Adaptive Policy Switching for Multi-Agent ASVs in Multi-Objective Aquatic Cleaning Environments.

Area of Science:

• Environmental Science
• Robotics
• Artificial Intelligence

Background:

• Plastic pollution poses a significant ecological threat to aquatic environments worldwide.
• Scalable and autonomous solutions are crucial for effective environmental cleanup operations.
• Coordinated multi-vehicle systems offer a promising approach to address large-scale pollution challenges.

Purpose of the Study:

• To develop a multi-agent reinforcement learning framework for coordinating Autonomous Surface Vehicles (ASVs).
• To decouple and optimize the conflicting goals of exploration and trash collection in ASV missions.
• To create an adaptive system capable of managing task switching for efficient autonomous cleanup.

Main Methods:

• Formulated the ASV coordination problem as a Partially Observable Markov Game.
• Implemented a multi-agent deep reinforcement learning framework using a shared Multitask Deep Q-network with convolutional backbone and dual heads.
• Developed an adaptive task-switching mechanism based on weighted reward aggregation and a reward-greedy strategy.

Main Results:

• The proposed framework demonstrated significant improvements in hypervolume (14%) and uniformity (300%) metrics compared to fixed-phase approaches.
• The system effectively adapted to various initial trash distributions, showcasing robust performance.
• Pareto fronts were constructed, capturing non-dominated solutions for exploration and cleaning task trade-offs.

Conclusions:

• The developed multi-agent deep reinforcement learning framework provides an effective and adaptive strategy for autonomous plastic cleanup in aquatic environments.
• Parameter sharing and egocentric state design enhance learning efficiency and enable experience aggregation across agents and tasks.
• This approach offers a scalable solution for environmental monitoring and remediation, providing decision-makers with diverse operational strategies.