Informed-Learning-Guided Visual Question Answering Model of Crop Disease

Area of Science:

• Agricultural Science
• Computer Vision
• Artificial Intelligence

Background:

• Current agricultural disease management relies on strategies for various disease stages, but decision-making is limited by single-image analysis.
• Existing visual question answering (VQA) models focus on disease species identification, neglecting crucial multi-attribute information and facing challenges with model structure and dataset biases.

Purpose of the Study:

• To develop an advanced VQA model for crop disease analysis that addresses limitations in existing methods.
• To improve decision-making in agriculture by considering multi-attribute visual characteristics of crop diseases.

Main Methods:

• Construction of the informed-learning-guided VQA model of crop disease (ILCD).
• Integration of coattention, multimodal fusion (MUTAN), and bias-balancing (BiBa) strategies within the ILCD model.
• Development of the Crop Disease Multi-attribute VQA with Prior Knowledge (CDwPK-VQA) dataset, incorporating prior knowledge to enhance visual attribute analysis.

Main Results:

• ILCD achieved accuracies of 68.90% on VQA-v2, 49.75% on VQA-CP v2, and 86.06% on the novel CDwPK-VQA dataset.
• Ablation studies confirmed the effectiveness of ILCD's coattention, MUTAN, and BiBa modules on the CDwPK-VQA dataset.
• ILCD demonstrated superior accuracy, performance, and value in agricultural applications.

Conclusions:

• The ILCD model, coupled with the CDwPK-VQA dataset, offers a significant advancement in agricultural disease diagnosis.
• The integrated approach effectively handles multi-attribute visual information, overcoming limitations of previous VQA models in agriculture.
• This research provides a robust framework for enhanced crop disease management through AI-driven visual analysis.