Complexity begets simplicity: self-supervised learning for palaeontological images with few or no labels

Palaeontology has seen widespread and growing use of machine learning (ML) to classify and analyse large datasets of fossils. However, palaeontology is a challenging field in which to apply ML. Datasets may be small or unlabelled, images may be complex and different from standard datasets and palaeontologists may lack specialist training and access to necessary computational resources. We show how these challenges can be addressed by using recent developments in self-supervised learning (SSL). Using a frozen DINOv3 feature extractor and a simple linear classifier, with reduced data, we can achieve comparable results to literature benchmarks using convolutional neural networks, the previous standard, when classifying fossil tracks, pollen, radiolaria, foraminifera and a dataset of diverse fossil images. In addition, the rich feature vectors generated by the model can be used for few-shot learning, unsupervised clustering, subject to the presence of imaging artefacts and quantification of disparity. Using state-of-the-art self-supervised methods increases accessibility by reducing code, compute and data required. It also maintains accuracy, while increasing reproducibility by reducing parameters.