Accurate weed detection in UAV grassland images via intelligent annotation and multi-dimensional network enhancement

Accurate weed detection from unmanned aerial vehicle (UAV) images remains difficult because weed targets often blend with surrounding vegetation, vary substantially in scale, and exhibit irregular boundaries under cluttered grassland backgrounds.

This study presents a UAV grassland weed detection framework that combines intelligent annotation with task-oriented detector adaptation. For label generation, SAM produces instance-level vegetation masks, SigLIP performs few-shot semantic matching, and a vision-language model audits ambiguous candidates using few-shot weed references, local candidate patches, and full-image context. Candidate outputs are converted into YOLO-format labels for detector training. For detection, a YOLOv11-based model is adapted with four complementary components: EIEStem for shallow boundary preservation, C3k2-EMA for multi-scale feature aggregation, SPPF-LSKA for contextual modeling, and LDConv for adaptive downsampling of irregular weed targets.

On 200 reference images, the SAM-SigLIP-VLM workflow achieved 92.4% label precision, a mean IoU of 0.813, and an F1-score of 0.905, while reducing the annotation time per image from 96.8 s to 31.6 s. Under five-fold image-level cross-validation on a UAV grassland weed dataset, the improved detector achieved 0.762 ± 0.004 mAP@0.5 and 0.545 ± 0.005 mAP@0.5:0.95, improving the YOLOv11 baseline by 4.0 and 4.7 percentage points, respectively.

Additional evaluations cover annotation quality, module ablation, module placement, downsampling mechanisms, detector comparison, and validation on an external public crop-weed dataset. These results show that the proposed framework improves both annotation efficiency and UAV grassland weed detection performance in complex vegetation backgrounds.