Cross-Modal and Contrastive Optimization for Explainable Multimodal Recognition of Predatory and Parasitic Insects

Area of Science:

• Agricultural Science
• Artificial Intelligence
• Ecology

Background:

• Conventional pest recognition struggles with environmental variations like lighting and occlusion.
• Accurate identification of natural enemies is crucial for ecological balance and pest suppression in agriculture.

Purpose of the Study:

• To develop a multimodal framework (MAVC-XAI) for enhanced natural enemy recognition and ecological interpretation in agricultural settings.
• To overcome limitations of vision-based methods in complex field conditions.

Main Methods:

• Utilized a dual-branch spatiotemporal feature extraction network for visual and acoustic signals.
• Implemented a cross-modal sampling attention mechanism for inter-modality alignment.
• Incorporated cross-species contrastive learning and an explainable generation module for ecological visualization.

Main Results:

• MAVC-XAI achieved high performance metrics: 0.938 accuracy, 0.932 precision, 0.927 recall, 0.929 F1-score, 0.872 mAP@50, and 97.8% Top-5 recognition.
• Outperformed unimodal and existing multimodal baseline models.
• Ablation studies confirmed the effectiveness of attention and contrastive learning modules.

Conclusions:

• The MAVC-XAI framework enables high-precision natural enemy identification in challenging ecological conditions.
• Provides an interpretable foundation for AI-driven ecological pest management and food security monitoring.