Thermal image segmentation in weedy fields via synthetic RGB-trained models and GAN-based cross-modality alignment.

Area of Science:

• Agricultural Science
• Computer Vision
• Remote Sensing

Background:

• Accurate plant segmentation in thermal imagery is crucial for high-throughput field phenotyping.
• Outdoor environments present challenges like low contrast and occlusions, hindering segmentation performance.

Purpose of the Study:

• To develop a framework for enhanced semantic segmentation in thermal images using synthetic data and generative models.
• To evaluate the impact of limited real annotations and cross-modality alignment on segmentation accuracy.

Main Methods:

• Leveraged synthetic RGB imagery and generative adversarial networks (GANs) for cross-modality alignment.
• Trained models on 1128 synthetic images and evaluated the integration of as few as 20 real annotated images.
• Utilized CycleGAN-Turbo for translating RGB to thermal imagery, enabling robust template matching.

Main Results:

• A maximum relative improvement of 25% in mean IoU score was observed when combining synthetic and limited real data compared to a synthetic-only baseline.
• Cross-domain translation via generative models facilitated robust template matching without calibration.
• The framework demonstrated significant performance boosts in complex field environments.

Conclusions:

• Combining synthetic data with limited manual annotations and cross-domain translation significantly enhances plant segmentation in thermal imagery.
• The proposed framework offers a viable solution for improving high-throughput field phenotyping in challenging outdoor conditions.
• Generative models and synthetic data are powerful tools for overcoming limitations in real-world agricultural image analysis.