The apple detection method based on multimodal features.

Area of Science:

• Computer Vision
• Agricultural Technology
• Robotics

Background:

• Accurate fruit detection is crucial for automated agriculture but challenged by lighting, occlusion, and clutter.
• Traditional RGB-based and incremental deep learning methods struggle with complex environmental variations.

Purpose of the Study:

• To develop an innovative apple detection method using multimodal feature fusion.
• To improve detection performance without altering core deep learning architectures.

Main Methods:

• Integrated four modalities: RGB images, color/edge maps, depth maps, and point clouds.
• Preprocessed point clouds using voxel sampling and anomaly detection for denoising and alignment.
• Redesigned YOLOv5 input layer for multi-channel feature fusion.

Main Results:

• Achieved 95.8% precision, 96.0% recall, and 95.9% F1-score in complex scenarios.
• Demonstrated significant precision improvements over RGB-only (7.4%) and RGB+depth (6.3%) methods.
• Multimodal fusion enhanced robustness against lighting variations and background noise.

Conclusions:

• Multimodal feature fusion offers a robust solution for fruit detection in challenging agricultural settings.
• The proposed method effectively leverages complementary data sources for improved detection accuracy.
• This approach advances automated harvesting and monitoring systems in agriculture.