Robust Fall Army Worm detection in maize using multimodal RGB and thermal image fusion.

Area of Science:

• Agricultural Science
• Computer Science
• Remote Sensing

Background:

• Fall Army Worm (FAW) is a major threat to global maize production, causing significant yield losses.
• Manual pest detection methods are labor-intensive, inefficient, and prone to errors, hindering effective crop management.
• Precision agriculture demands accurate and automated tools for early pest and disease identification.

Purpose of the Study:

• To develop and evaluate a novel deep learning framework for automatic classification of FAW-infested maize crops.
• To enhance detection accuracy by integrating RGB and thermal imaging modalities through multimodal fusion.
• To compare the performance of fused models against single-modality and unfused deep learning approaches.

Main Methods:

• A hybrid deep neural network-Vision Transformer (DNN-ViT) model was designed for multimodal image fusion.
• Two fusion strategies were explored: feature-level fusion (CNN features + DNN) and image-level fusion (6-channel RGB-thermal + ViT).
• Performance was evaluated using accuracy, precision, recall, F1-score, and AUC-ROC on a test dataset.

Main Results:

• The fused DNN-ViT model achieved high performance with 0.98 accuracy, precision, recall, F1-score, and AUC-ROC.
• Multimodal fusion significantly outperformed models trained on RGB-only or thermal-only data.
• An ablation study confirmed the effectiveness of fusion, showing a substantial performance drop (accuracy 0.60) without it.

Conclusions:

• Integrating RGB and thermal imagery via deep learning fusion offers a robust solution for accurate FAW detection in maize.
• The proposed multimodal framework enhances crop health monitoring capabilities in precision agriculture.
• Future work should focus on advanced fusion techniques and field deployment for practical application.