An end-to-end dual backbone framework for UAV fish disease detection using optimized GateHCO feature fusion mechanism

Early detection of fish disease is of great significance to inhibit the transmissible disease. Nowadays, computer vision and deep learning methods serve as the most innovative method to identify fish disease. However, most vision technologies to detect fish disease are limited to 2D objects which focus on the feature extraction from images. The dynamic information in the videos are not fully utilized. In this study, we developed a novel hybrid deep learning method and established a dual-backbone architecture for fish disease detection. The model can both extract static appearance features and dynamic behavioral features simultaneous. In particular, we designed a GateHCO (Gated Heat Conduction) attention module, inspired by the heat conduction process and simulating the information propagation in the data, which achieve an effective cross-modal information interaction at a relatively low computation cost. In addition, we developed a UAV (unmanned aerial vehicle) to collect experimental datasets to validate the prototype method detecting fish with GCHD (grass carp haemorrhage disease). In our collected datasets by the UAV, the proposed hybrid method achieved 93.64 mAP, 55.33 ms, 25.55 G FLOPs. The evaluation indicators are notably superior than general detection models (including YOLOv7, YOLOv10, YOLOv12, etc.), RT-DETR and one-stage spatio-temporal framework. We also confirm its superiority than 3D CNN architectures based on ResNet, ResNeXt, and ShuffleNetV2 as well as other advanced Transformer-based architectures such as the MViT series.