Simulation of Automatically Annotated Visible and Multi-/Hyperspectral Images Using the Helios 3D Plant and Radiative...

Area of Science:

• Plant science
• Remote sensing
• Computer graphics

Background:

• Supervised deep learning models for plant trait analysis require extensive annotated datasets.
• Manual data collection for these datasets is labor-intensive and time-consuming.
• Extracting complex plant traits from remote sensing data remains challenging.

Purpose of the Study:

• To develop a radiative transfer modeling framework for simulating plant images with accurate annotations.
• To reduce reliance on manually collected and annotated datasets for plant trait analysis.
• To enable unsupervised learning for plant trait extraction using simulated data.

Main Methods:

• Utilized the Helios 3D plant modeling software to create a radiative transfer modeling framework.
• Simulated various imaging modalities including RGB, multi-/hyperspectral, thermal, and depth cameras.
• Generated 3D plant and soil models with random variations and specified properties, explicitly modeling radiation transfer physics.

Main Results:

• The framework successfully generated high-quality, labeled synthetic plant images under various lighting conditions.
• Simulated images possess fully resolved reference labels for physical, chemical, and physiological plant traits.
• Demonstrated the framework's capability to support unsupervised learning by training models exclusively on simulated data.

Conclusions:

• The proposed Helios-based framework effectively generates synthetic, annotated plant images, significantly reducing the need for manual data annotation.
• This approach facilitates the application of deep learning models in plant trait analysis, particularly for unsupervised learning tasks.
• The framework provides a viable solution for overcoming data acquisition bottlenecks in plant remote and proximal sensing.