deepNIR: Datasets for Generating Synthetic NIR Images and Improved Fruit Detection System Using Deep Learning...

Area of Science:

• Computer Vision
• Machine Learning
• Remote Sensing

Background:

• High-quality datasets are crucial for the generalization and deployment of data-driven deep neural networks.
• Synthetic data generation, particularly for Near-Infrared (NIR) imaging, often requires extensive training samples.
• Existing public datasets may not fully cater to the specific needs of advanced agricultural applications.

Purpose of the Study:

• To present and release novel NIR+RGB datasets for synthetic NIR image generation.
• To provide comprehensive bounding-box level fruit detection datasets for machine learning models.
• To establish a baseline for future research in agricultural computer vision and remote sensing.

Main Methods:

• Reprocessing and expanding public datasets (nirscene, SEN12MS) with oversampling and digital number (DN) to pixel value conversion.
• Collecting and curating a new NIR+RGB sweet pepper (capsicum) dataset from commercial farms.
• Generating manual annotations for 11 fruit types, including novel additions like blueberry and kiwi, resulting in 11 new bounding box datasets.

Main Results:

• Achieved competitive Frechet Inception Distances (FIDs) for synthetic NIR image generation (e.g., 11.36 for nirscene1).
• Released a combined dataset with 162,000 bounding box instances across 11 fruit categories.
• Demonstrated strong performance using Yolov5, achieving mean-average-precision (mAP) scores up to 0.812 for fruit detection.

Conclusions:

• The released NIR+RGB datasets are suitable for synthetic NIR image generation tasks.
• The comprehensive fruit bounding box datasets provide a valuable resource for training and evaluating fruit detection models.
• These datasets are expected to serve as a foundational resource for advancing agricultural technology and research.