ConvDeiT-Tiny: Adding Local Inductive Bias to DeiT-Ti for Enhanced Maize Leaf Disease Classification

Area of Science:

• Computer Vision
• Plant Pathology
• Machine Learning

Background:

• Accurate maize leaf disease identification is crucial for crop yield preservation, especially where expert access is limited.
• Vision transformers (ViTs) show promise in image recognition but struggle with fine-grained classification due to limited local texture modeling.
• Existing ViTs lack the necessary inductive bias for detailed analysis of plant disease symptoms.

Purpose of the Study:

• To develop a lightweight hybrid vision transformer model for improved maize leaf disease classification.
• To enhance the modeling of local texture patterns within transformer architectures.
• To provide a more accurate and accessible tool for farmers to identify maize diseases.

Main Methods:

• Proposed ConvDeiT-Tiny, a hybrid ViT integrating depthwise convolutions with multi-head self-attention in early transformer blocks.
• Fused local features from convolutions and global features from attention to create richer token representations.
• Evaluated model performance across three distinct maize leaf disease datasets (CD&S, primary, Kaggle).

Main Results:

• ConvDeiT-Tiny (6.9 M parameters) consistently outperformed standard DeiT models (e.g., DeiT-Ti, DeiT-S) when trained from scratch.
• Achieved high accuracy with transfer learning: 99.15% (CD&S), 99.35% (primary), and 98.60% (Kaggle).
• Demonstrated superior performance with significantly fewer parameters compared to previous studies.

Conclusions:

• Injecting local inductive bias via convolutions into early transformer blocks significantly benefits maize leaf disease classification.
• ConvDeiT-Tiny offers a computationally efficient and highly accurate solution for automated maize disease identification.
• The model's explainability features aid in understanding disease lesion identification, enhancing trust and utility for agricultural applications.