Taxonomy-informed hierarchical deep learning for otolith image identification in coral reef fishes

Existing studies on automatic fish otolith identification have predominantly adopted classification frameworks that use species as the sole output level, limiting the effective use of hierarchical information embedded in fish taxonomy. In this study, using otolith images of coral reef fishes collected from the Qilianyu Islands, Xisha Islands, China, during 2020-2025, we developed a hierarchical deep learning model incorporating taxonomic constraints and compared it with a non-hierarchical baseline model. Both models were trained, validated and tested on otolith images from 25 species, and the generalization performance of the hierarchical model was further evaluated on 33 species not included in model training. The results showed that the hierarchical model achieved higher recognition accuracy than the non-hierarchical model at the family, genus and species levels, with species-level accuracy improving from 94.80% to 98.00%. Both micro-averaged and macro-averaged metrics also improved, and misclassifications across families and genera were markedly reduced. UMAP and Grad-CAM visualizations showed that the hierarchical model learned discriminative embedding features with clear species-level separation, and that its high-response regions were mainly concentrated along the otolith contour and local structural features. External validation further showed that, under unseen-species conditions, the hierarchical model achieved strong species recognition with only a small number of support samples. Overall, these results show that taxonomic hierarchy constraints can improve otolith image identification and support the development of a biologically informed framework that is extensible to unseen species.