Related Experiment Video
Updated: Aug 29, 2026

Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
DEEPFIN: A deep learning tool for fish image classification from unlabeled data
Alexandru Mihai1, Billy Moore1, Marleen Klann2
1Marine Climate Change Unit, Okinawa Institute of Science and Technology (OIST), Onna, Okinawa, Japan.
Abstract:
Automating fish image classification would relieve a growing annotation bottleneck in ecology, developmental biology, and fisheries science, where imagery accumulates far faster than experts can label it. We present DEEPFIN, an annotation-free image analysis pipeline that combines foreground segmentation, frozen pretrained convolutional encoders, non-linear dimensionality reduction, and density-based clustering. DEEPFIN recovers the seven canonical developmental stages of the clownfish Amphiprion ocellaris with 87% cluster-to-class accuracy and separates three congeneric anemonefish species from images alone. Independent validation across three public datasets spanning in situ underwater, multi-condition, and controlled ground-truth-masked imaging yields consistent pairwise species discrimination without labels, matching or approaching contemporary supervised methods on the same data. Beyond replicating expert categories, the pipeline surfaces intra-class variation that fixed-label classifiers cannot expose. DEEPFIN is a lightweight, generalizable framework for extracting biological structure from the growing volumes of unlabeled fish imagery.