Refer-ASV: Referring Multi-Object Tracking in Autonomous Surface Vehicle Navigation Scenes

Area of Science:

• Robotics and Computer Vision
• Maritime Autonomous Systems

Background:

• Water-surface perception is crucial for autonomous surface vehicle (ASV) navigation.
• Existing referring multi-object tracking (RMOT) benchmarks lack suitability for complex maritime environments.

Purpose of the Study:

• To introduce the first RMOT dataset (Refer-ASV) specifically designed for ASV navigation in challenging water-surface scenes.
• To propose a novel baseline framework (RAMOT) for enhanced visual-language alignment and robustness in maritime RMOT.

Main Methods:

• Construction of the Refer-ASV dataset from real-world ASV videos, featuring diverse navigation scenarios and detailed vessel classifications.
• Development of RAMOT, an end-to-end framework integrating improved visual-language alignment for robust tracking in maritime settings.

Main Results:

• The proposed RAMOT framework achieved a HOTA score of 39.97 on the Refer-ASV dataset.
• RAMOT demonstrated superior performance compared to existing methods in challenging maritime environments.
• Experiments on the Refer-KITTI dataset confirmed RAMOT's generalization capabilities across different scenes.

Conclusions:

• Refer-ASV addresses the critical need for specialized RMOT benchmarks in ASV navigation.
• RAMOT provides a significant advancement in robust visual-language-based object tracking for maritime applications.
• The developed dataset and framework pave the way for more reliable autonomous navigation systems in complex water-surface environments.