Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

Efficient Image-Only Inference for Multimodal Crop Disease Recognition via Modal Dropout and Adaptive Multi-Task Loss

Jianlin Qiu¹, Depeng Gao¹, Shuxi Chen¹

¹School of Yonyou Digital Intelligence, Nantong Institute of Technology, Nantong 226002, China.

Sensors (Basel, Switzerland)

|July 15, 2026

Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

MS-YOLOv11: A multi-scale feature fusion network for real-time rooftop photovoltaic detection from UAV images.

PloS one·2026

Same author

MCRBM-CNN: A Hybrid Deep Learning Framework for Robust SSVEP Classification.

Sensors (Basel, Switzerland)·2025

Same author

Image super-resolution method using a generative adversarial network incorporating attention and residual density.

Scientific reports·2025

Same author

Effect of TRIM67 knockout on inflammation and macrophage survival in Salmonella Typhimurium-infected mice.

Archives of microbiology·2025

Same author

RGB-FIR Multimodal Pedestrian Detection with Cross-Modality Context Attentional Model.

Sensors (Basel, Switzerland)·2025

Same author

Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition.

Sensors (Basel, Switzerland)·2025

This study introduces MTL-AWL, a framework enabling faster crop disease diagnosis by using vision-language models (VLMs) for training but image-only analysis during real-time field use.

Area of Science:

Agricultural Science
Computer Vision
Machine Learning

Background:

Crop leaf diseases result in significant annual yield losses (10-40%).
Current field diagnosis methods struggle with timely and accurate identification.
Vision-language models (VLMs) improve recognition but are too slow for real-time field application due to text processing requirements.

Purpose of the Study:

To develop a framework (MTL-AWL) for efficient, real-time crop disease diagnosis.
To enable image-only deployment of VLMs at multimodal accuracy for field use.
To overcome the speed limitations of traditional multimodal pipelines in agricultural settings.

Main Methods:

Implemented a training-inference asymmetry where VLM text is used for supervision during training.

Keywords:

adaptive weight learning crop leaf disease recognition modal dropout multi-task loss function

Related Experiment Videos

Utilized coupled mechanisms for retaining VLM semantics in the image encoder for image-only inference.

Employed a modal-dropout strategy during training to enhance independent cross-modal representation learning.

Applied an adaptive multi-task loss optimizing contrastive alignment, attention diversity, and modality consistency.

Main Results:

The MTL-AWL framework achieved 818 FPS at inference, 3.7x faster than multimodal methods, with a minimal 0.41% accuracy cost.
Image-only deployment reached high accuracy: 99.30% on soybean and 72.65% on PlantDoc.
Cross-modal alignment was identified as the primary mechanism for VLM knowledge transfer, with contrastive weights dominating the loss function.

Conclusions:

MTL-AWL enables real-time, offline field screening for crop diseases.
The framework successfully bridges the gap between VLM accuracy and the speed required for practical field applications.
This approach offers a computationally efficient solution for accurate plant disease detection.