In-domain versus out-of-domain transfer learning in plankton image classification.

Area of Science:

• Marine biology and ecology
• Computational biology and machine learning

Background:

• Plankton are crucial to aquatic food webs and can serve as biosensors due to their sensitivity to environmental changes.
• High-resolution imaging systems generate vast plankton datasets, but annotation is costly and datasets are often imbalanced.
• Transfer learning, particularly using ImageNet, is a common solution for data scarcity and imbalance in plankton image classification.

Purpose of the Study:

• To compare the effectiveness of in-domain versus out-of-domain transfer learning for plankton image classification.
• To evaluate different transfer learning pipelines, including pre-training from scratch on plankton data, natural image datasets, and a two-stage fine-tuning approach.
• To assess the performance of state-of-the-art models like Vision Transformers and ConvNeXt, and their ensembles for plankton classification.

Main Methods:

• Designed three transfer learning pipelines: (1) pre-training from scratch on plankton data, (2) pre-training on ImageNet1K/22K, and (3) two-stage fine-tuning (ImageNet -> plankton dataset -> target dataset).
• Fine-tuned pre-trained models on three benchmark plankton datasets.
• Evaluated three ImageNet22K pre-trained Vision Transformers and one ConvNeXt model.
• Created and tested an ensemble of these advanced models.

Main Results:

• Out-of-domain pre-training with ImageNet22K outperformed in-domain plankton pre-training, yielding an average 6% increase in test accuracy.
• Single models (Vision Transformers and ConvNeXt) achieved results comparable to or better than existing state-of-the-art CNN ensembles.
• The developed ensemble model surpassed the state-of-the-art performance on all three plankton benchmark datasets.

Conclusions:

• Out-of-domain transfer learning, specifically using ImageNet22K, is highly effective for plankton image classification, addressing data imbalance and annotation costs.
• Advanced deep learning architectures like Vision Transformers and ConvNeXt, especially when ensembled, represent the current state-of-the-art for plankton image classification.
• The open-source code facilitates further research and development in plankton image analysis.