DIC-Transformer: interpretation of plant disease classification results using image caption generation technology

Area of Science:

• Agricultural Science
• Computer Vision
• Artificial Intelligence

Background:

• Current agricultural disease image classification systems lack explanatory capabilities for disease characteristics.
• There is a need for models that can both classify diseases and provide descriptive insights into their visual features.

Purpose of the Study:

• To develop an advanced model for agricultural disease image classification and interpretation.
• To generate descriptive captions for plant disease images, enhancing understanding of disease characteristics.

Main Methods:

• Proposed a two-stage model, DIC-Transformer, integrating detection, interpretation, and classification.
• Utilized Faster R-CNN with Swin Transformer for disease area detection and feature extraction.
• Employed a Transformer-based approach for generating image captions and weighted feature vectors for improved classification.

Main Results:

• The DIC-Transformer achieved superior performance in both image captioning and classification tasks compared to existing models.
• Achieved captioning metrics (BLEU-1, CIDEr-D, ROUGE) of 0.756, 450.51, and 0.721, outperforming the best comparison model.
• Attained classification accuracy, recall, and F1 score of 0.854, 0.854, and 0.853, respectively, exceeding the best comparison model.

Conclusions:

• The DIC-Transformer model demonstrates significant advancements in agricultural disease image analysis.
• The model's ability to generate descriptive captions alongside accurate classification offers valuable insights for disease management.
• The developed ADCG-18 dataset supports further research in agricultural disease image understanding.