FFM-ViT: an efficient fish species classification method based on deep features and transformers

Morphological identification of fish species plays a crucial role in the monitoring and management of fishery resources and biodiversity conservation. However, existing classification methods fail to meet practical needs when confronted with small fish data sets and high similarity. In this paper, we propose a novel deep learning method called feature fusion module vision transformer (FFM-ViT). The essence of FFM-ViT lies in abandoning the direct patch operation used in traditional vision transformer (ViT) and introducing Mobile Inverted Bottleneck Convolution (MBConv) and Fused Mobile Inverted Bottleneck Convolution (Fuse-MBConv) blocks to obtain more accurate high-dimensional information. To enhance feature extraction capability and channel feature fusion, we also introduce the channel spatial merge attention (CSMA) module. Furthermore, we have curated a dataset consisting of 78 categories named Oceanfish78. Our model achieves an impressive accuracy rate of 90.2% on this dataset, surpassing the 80.4% accuracy achieved by the ViT model without pre-trained weights significantly. Additionally, we conducted tests on several datasets, such as fish4knowledge and Fish31, while comparing our proposed method with other deep learning models, including shufflenet, convnext and swin transformer, through comprehensive empirical analysis. The results demonstrate that our proposed method outperforms existing approaches comprehensively, not only providing an effective solution for fish classification, but also offering valuable insights for approximate target recognition in other environments.