Structure-Driven Feature Modeling for Robust Fish Detection in Complex Underwater Environments

Accurate fish detection in complex underwater environments remains challenging due to scattering, low contrast, blur, occlusion, and background interference, which significantly degrade appearance-based representations. To address these issues, this study develops a progressive Structure-Driven Feature Modeling mechanism for underwater fish detection. The proposed mechanism extracts multi-scale smoothed residuals from shallow features, normalizes their responses, and uses the resulting structural representation to guide spatial feature modulation, while retaining complementary appearance information through feature fusion. To implement this framework, a set of Structure-Aware Feature Modeling Units is designed, integrating Spectral Encoder, Structure Channel Attention, Multi-Scale Structure Difference, Structure Normalization Module, Structure-Guided Spatial Attention, and Refine. These modules collaboratively enhance structural representations across channel and spatial dimensions. Furthermore, a Structure-Driven Feature Modeling method and a Structure-Driven Fusion strategy are introduced to facilitate progressive multi-scale structural learning based on the proposed Structure-Aware Feature Modeling Units, while enabling complementary interaction between structural and appearance features. To support evaluation, a challenging underwater fish detection dataset, Medaka, is constructed, covering occlusion, illumination degradation, and complex background interference. Extensive controlled experiments demonstrate that the proposed method can be integrated into multiple YOLO-based architectures and consistently improves detection performance under the same dataset partition and evaluation protocol. Specifically, on YOLOv8n, mAP@0.5 increases from 0.6332 to 0.6647 (+3.15 pp), and mAP@0.5:0.95 improves from 0.2495 to 0.2691 (+1.96 pp), while Precision and Recall are enhanced by 7.55 pp and 3.80 pp, respectively. Consistent gains are also observed on YOLOv5n, YOLOv7-tiny, YOLOv10n, YOLOv11n, and YOLOv12n, supporting the effectiveness and cross-architecture transferability of the proposed Structure-Driven Feature Modeling mechanism within the evaluated YOLO family. Additional evaluation on the public Blue Bot Dataset showed that Structure-Driven YOLOv8n improved mAP@0.5 and mAP@0.5:0.95 over YOLOv8n by 3.36 pp and 1.85 pp, respectively. Overall, the proposed framework provides an effective solution for structure-aware feature learning in the evaluated underwater detection settings, particularly for challenging cases involving small objects, blurred boundaries, occlusions, and visually similar targets.