A two-stage OOD-aware approach for mosquito species detection based on RT-DETRv2

Mosquitoes are important vectors of infectious diseases, and accurate species identification is essential for effective surveillance and control. Traditional image-based identification methods are labor-intensive, while existing deep learning models often exhibit limited generalization in real-world environments due to sensitivity to out-of-distribution (OOD) samples. To address these challenges, this study constructs a real-world mosquito dataset by integrating multiple publicly available sources, comprising 11,514 original images across 33 predefined species. To mitigate class imbalance, data augmentation is applied to the training set, resulting in a total of 17,302 images for training and evaluation. In addition, two OOD datasets are constructed, including a non-target mosquito dataset and a non-Culicidae insect dataset. Based on a comprehensive comparison of mainstream object detection models, RT-DETRv2 with a PResNet-101 backbone is selected as the baseline detector. Building upon this model, a two-stage OOD-aware approach, termed MosquitoID, is proposed by integrating Mahalanobis distance and Energy-based scoring. The first stage performs image-level OOD filtering using Mahalanobis distance, while the second stage conducts instance-level OOD discrimination based on Energy scores. Under a fixed threshold setting determined from the ID validation set to retain 95% of ID samples, experimental results show that the proposed method achieves a Precision of 92.0%, Recall of 93.8%, mAP[Formula: see text] of 90.4%, and mAP[Formula: see text] of 84.1% on the ID dataset. For OOD detection, MosquitoID attains AUROC values of 0.914 and 0.935 on the two OOD datasets, with corresponding FPR@95TPR values of 0.205 and 0.156, and [Formula: see text]-scores of 83.7% and 90.8%. These results indicate that the proposed approach improves robustness to unknown samples and provides a practical method for mosquito detection in real-world surveillance scenarios.