MulTNet: A Multi-Scale Transformer Network for Marine Image Segmentation toward Fishing

Area of Science:

• Computer Vision
• Marine Robotics
• Artificial Intelligence

Background:

• Accurate image segmentation is crucial for autonomous underwater vehicles (AUVs) in marine applications.
• Existing segmentation methods struggle with low-quality, complex underwater imagery, hindering precise marine animal identification.
• This limitation impacts the efficiency and effectiveness of automated marine product capture.

Purpose of the Study:

• To introduce a novel Multi-Scale Transformer Network (MulTNet) for improved marine animal image segmentation.
• To combine the strengths of Convolutional Neural Networks (CNNs) and transformers for enhanced feature extraction and contextual understanding.
• To address the computational challenges associated with deep learning models in underwater sensing.

Main Methods:

• A dimensionality reduction CNN module (DRCM) progressively extracts low-level features.
• A multi-scale transformer module (MTM) with parallel small-scale encoders and a large-scale transformer layer captures multi-scale contextual information.
• The network integrates CNNs for feature extraction and transformers for contextual reasoning.

Main Results:

• MulTNet demonstrated superior performance compared to existing advanced image segmentation networks.
• Significant improvements in Mean Intersection over Union (mIOU) were observed: 0.76% on a marine animal dataset and 0.29% on the ISIC 2018 dataset.
• The model effectively handles the complexities of underwater image segmentation.

Conclusions:

• MulTNet offers a robust solution for segmenting marine animals in challenging underwater environments.
• The proposed method holds significant application value for improving AUV-based fishing and marine monitoring systems.
• This research advances the capabilities of computer vision in marine robotics.