Surface Vessels Detection and Tracking Method and Datasets with Multi-Source Data Fusion in Real-World Complex...

Area of Science:

• Maritime technology
• Autonomous systems
• Sensor fusion

Background:

• Single-modality sensors (AIS, radar, visible) have limitations for intelligent ship navigation.
• Challenges include low sensor frequency, radar blind zones, and lack of depth information.
• Accurate environment sensing is crucial for safe autonomous navigation.

Purpose of the Study:

• To propose a multi-source data fusion framework integrating AIS, radar, and visible data.
• To enhance sensing performance in complex marine environments and adverse weather.
• To improve the accuracy and identity consistency of object detection and tracking.

Main Methods:

• Developed a multi-stage detection and tracking method (MSTrack) with feedback loops.
• Implemented a cascade association matching method for multi-source trajectory association.
• Created the first multi-source fusion dataset for intelligent vessels (WHUT-MSFVessel).

Main Results:

• MSTrack refines detections by feeding historical fusion results back to earlier stages.
• Cascade association matching reduces errors from measurement noise and projection systems.
• Achieved high MOTA scores (0.872 radar, 0.938 visible) and IDF1 scores (0.811, 0.929).
• Fusion accuracy reached up to 0.9.

Conclusions:

• Multi-source data fusion significantly improves sensing accuracy and tracking identity consistency.
• The proposed framework provides comprehensive environmental perception for safer intelligent vessel navigation.
• The WHUT-MSFVessel dataset facilitates further research in this domain.