A unified RT-DETR-Mask2Former framework for segmentation-guided plant leaf disease detection and classification in real-world agricultural environments

Plant leaf disease image analysis under real-world field conditions remains challenging due to cluttered backgrounds, illumination variation, overlapping leaves, scale changes, and irregular lesion patterns. This study presents a unified RT-DETR-Mask2Former framework for segmentation-guided plant leaf disease detection and classification. In the proposed pipeline, RT-DETR first localizes leaf regions from complex agricultural images, after which Mask2Former generates fine-grained lesion masks within the detected regions. The refined lesion masks are used to isolate disease-affected regions, enabling EfficientNet-B4 to extract disease-focused feature representations for classification. An EMA-normalized adaptive task-weighting procedure was applied with AdamW optimization to modulate the relative gradient scales of the separately supervised modules. Model development and internal evaluation were conducted using leakage-controlled, group-aware PlantVillage (PV) partitions, whereas 4,200 field-acquired images were reserved exclusively for independent external evaluation. On the complete 38-category PV internal test, accuracy and macro-F1 were 98.34% and 97.97%. On the nine-category source-matched subset, accuracy and macro-F1 were 97.43% and 96.77%, versus 84.93% and 80.80% in the field. PV-to-field performance declined from 95.63% to 89.28% for leaf-localization AP@0.5 and from 92.47%/86.00% to 77.90%/63.80% for lesion-positive Dice/IoU. Lesion guidance improved field macro-F1 from 73.97% to 80.80%. The complete pipeline required approximately 285 GFLOPs and 63 ms per image (16 FPS), demonstrating a practical accuracy-efficiency trade-off while retaining a measurable field-domain gap.