Leveraging synthetic data produced from museum specimens to train adaptable species classification models.

Area of Science:

• Ecology
• Computer Vision
• Machine Learning

Background:

• Computer vision models require extensive annotated data for ecological research, posing a significant challenge due to resource demands.
• Training computer vision models with synthetic images often results in poor generalization to real-world photographs.

Purpose of the Study:

• To present a modular pipeline for training generalizable computer vision classification models using synthetic images.
• To address the challenge of limited annotated data in ecological research by leveraging synthetic data generation.

Main Methods:

• Developed a pipeline involving 3D asset creation (using 3D scanners), synthetic image generation (using open-source graphics software), and domain-adaptive classification model training.
• Applied the pipeline to classify skulls of 16 mammal species (order Carnivora).
• Evaluated domain adaptation techniques including Maximum Mean Discrepancy (MMD) loss, fine-tuning, and data supplementation.

Main Results:

• Improved classification accuracy on real photographs from 55.4% to a maximum of 95.1% using the proposed pipeline and domain adaptation.
• Qualitative analysis using t-distributed stochastic neighbor embedding (t-SNE) and gradient-weighted class activation mapping (Grad-CAM) compared domain adaptation techniques.

Conclusions:

• The pipeline demonstrates the feasibility of using synthetic images for ecological computer vision applications.
• Highlights the potential of 3D assets and museum specimens for scalable and generalizable model training in ecology.