Using deep learning models to decode emotional states in horses

In this study, we explore machine learning models for predicting emotional states in ridden horses. We manually label the images to train the models in a supervised manner. We perform data exploration and use different cropping methods, mainly based on Yolo and Faster R-CNN, to create two new datasets: 1) the cropped body, and 2) the cropped head dataset. We train various convolutional neural network (CNN) models on both cropped and uncropped datasets and compare their performance in emotion prediction of ridden horses. Despite the cropped head dataset lacking important regions like the tail (commonly annotated by experts), it yields the best results with an accuracy of 87%, precision of 79%, and recall of 97%. Furthermore, we update our models using various techniques, such as transfer learning and fine-tuning, to further improve their performance. Finally, we employ three interpretation methods to analyze the internal workings of our models, finding that LIME effectively identifies features similar to those used by experts for annotation.