Efficient Image-Only Inference for Multimodal Crop Disease Recognition via Modal Dropout and Adaptive Multi-Task Loss...

Area of Science:

• Agricultural Science
• Computer Vision
• Machine Learning

Background:

• Crop leaf diseases result in significant annual yield losses (10-40%).
• Current field diagnosis methods struggle with timely and accurate identification.
• Vision-language models (VLMs) improve recognition but are too slow for real-time field application due to text processing requirements.

Purpose of the Study:

• To develop a framework (MTL-AWL) for efficient, real-time crop disease diagnosis.
• To enable image-only deployment of VLMs at multimodal accuracy for field use.
• To overcome the speed limitations of traditional multimodal pipelines in agricultural settings.

Main Methods:

• Implemented a training-inference asymmetry where VLM text is used for supervision during training.
• Utilized coupled mechanisms for retaining VLM semantics in the image encoder for image-only inference.
• Employed a modal-dropout strategy during training to enhance independent cross-modal representation learning.
• Applied an adaptive multi-task loss optimizing contrastive alignment, attention diversity, and modality consistency.

Main Results:

• The MTL-AWL framework achieved 818 FPS at inference, 3.7x faster than multimodal methods, with a minimal 0.41% accuracy cost.
• Image-only deployment reached high accuracy: 99.30% on soybean and 72.65% on PlantDoc.
• Cross-modal alignment was identified as the primary mechanism for VLM knowledge transfer, with contrastive weights dominating the loss function.

Conclusions:

• MTL-AWL enables real-time, offline field screening for crop diseases.
• The framework successfully bridges the gap between VLM accuracy and the speed required for practical field applications.
• This approach offers a computationally efficient solution for accurate plant disease detection.