Acoustic-optical joint underwater object detection with multi-modality correlation features matching network.

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Marine Technology

Background:

• Underwater object detection is crucial for marine exploration but faces challenges due to weak alignment between optical and acoustic (sonar) sensors.
• Existing research lacks robust multi-modality approaches for integrating underwater camera and sonar data.

Purpose of the Study:

• To develop a novel acoustic-optical joint multi-modality detection model (AOMNet) for enhanced underwater object detection.
• To address the challenges of weak image alignment and improve detection accuracy using complementary sensor data.

Main Methods:

• Proposed the AOMNet model integrating features from underwater cameras and sonar devices.
• Introduced Cross-Feature and Predicted Position (CPFM) for optimizing candidate region generation using contour and texture similarity.
• Implemented a multi-modality registration and region matching strategy (PLFM+BRM) with morphological calculation and affine transformation for accurate bounding box regression.

Main Results:

• The AOMNet model demonstrated superior performance compared to single-modality and existing multi-modality detection models.
• Achieved a mean Average Precision (mAP) of 81.3% on a self-collected dataset of sonar and underwater camera images.
• Validated the effectiveness of CPMFM and PLFM+BRM strategies in improving detection accuracy.

Conclusions:

• AOMNet offers a significant advancement in underwater object detection by effectively fusing acoustic and optical data.
• The proposed feature-level and decision-level strategies enhance the robustness and accuracy of multi-modality detection in challenging underwater environments.