DeepFruits: A Fruit Detection System Using Deep Neural Networks

Area of Science:

• Agricultural technology
• Computer vision
• Deep learning

Background:

• Accurate fruit detection is crucial for autonomous agricultural robots, impacting yield estimation and harvesting.
• Existing object detection methods require significant annotation effort.

Purpose of the Study:

• To develop an accurate, fast, and reliable fruit detection system for agricultural robotics.
• To adapt and enhance the Faster Region-based CNN (Faster R-CNN) model for multi-modal fruit detection.

Main Methods:

• Utilized deep convolutional neural networks, specifically adapting the Faster R-CNN model.
• Employed transfer learning with multi-modal imagery (RGB and Near-Infrared - NIR).
• Explored early and late fusion techniques to combine RGB and NIR data.

Main Results:

• Achieved state-of-the-art results in fruit detection, improving the F1 score from 0.807 to 0.838 for sweet pepper detection.
• Demonstrated a significant reduction in annotation time by using bounding box annotation instead of pixel-level annotation.
• Successfully retrained the model for detecting seven different fruit types within a four-hour timeframe per fruit.

Conclusions:

• The novel multi-modal Faster R-CNN model offers improved accuracy and efficiency for fruit detection.
• The bounding box annotation method drastically reduces the effort required for deploying the system to new fruits.
• This approach represents a significant advancement for autonomous agricultural platforms.