Automated whale shark recognition and tracking using drones and deep learning

Whale sharks (Rhincodon typus) are migratory species known to form temporary coastal aggregations in regions of high plankton density, such as Bahía de los Ángeles (BLA), Mexico. Due to various anthropogenic threats, the species is classified as vulnerable, underscoring the need for effective and non-invasive monitoring strategies. This study explores the integration of drone-based remote sensing with deep learning (DL) techniques for the automated recognition and tracking of whale sharks. In October 2023, 59 drone flights-42 from boats and 17 from land-were conducted, resulting in 23 recorded sightings. Two DL-based approaches were implemented: DeepLabCut (DLC), based on pose estimation, and multi-scale patch (MSP), based on image region classification. Both strategies employed convolutional neural networks, with data augmentation and transfer learning techniques to enhance performance. The most effective model, MSP with multi-size and overlapping patches, achieved a macro F1 score of 0.91 and a covered area metric of 0.65. Statistical analysis indicated that environmental and operational factors-including turbidity, solar glare and external agents-had minimal impact on predictive performance. These results highlight the potential of DL-enhanced drone monitoring for scalable, accurate and non-invasive recognition and tracking of whale sharks, supporting ecological research and conservation efforts in BLA and comparable marine environments.