Acoustic-optical joint underwater object detection with multi-modality correlation features matching network

Underwater object detection is of significant importance for exploring underwater world, however, there is little research focusing on multi-modality detection in this field. Due to weak image alignment of underwater camera and sonar device, there is big challenge for object detection with underwater acoustic-optical images. On the basis of images from underwater camera and sonar device, this paper proposes a novel acoustic-optical joint multi-modality detection model (AOMNet in short) for underwater object detection. At the feature level of AOMNet model, we propose CPMFM based on cross-feature and predicted position, which utilizes contour and texture similarity between different modalities, as well as the predicted position parameters, to optimize the generation strategy of candidate regions and guide the model to generate more accurate bounding frames. At the decision-making level of AOMNet model, we use PLFM+BRM strategy to register multi-modality, and implement region matching through morphological calculation and affine transformation of mask to ensure more accurate boundary box regression. Finally, our method is validated on self-collected dataset of sonar and underwater camera. The proposed AOMNet algorithm shows significant advantages over existing single and multi-modality detection models, with mAP of 81.3%.