LeafGen: Structure-aware Leaf Image Generation for Annotation-free Leaf Instance Segmentation.

Area of Science:

• Computer Vision
• Plant Science
• Machine Learning

Background:

• Instance segmentation of plant leaves is vital for plant phenotyping, but manual annotation of training data is labor-intensive.
• Pre-trained segmentation models (e.g., Segment Anything) offer zero-shot capabilities but struggle with plant-specific challenges like occlusion and texture.
• Existing foundation models lack sufficient plant imagery, leading to suboptimal performance in leaf segmentation tasks.

Purpose of the Study:

• To develop a fully automatic method for generating training datasets for leaf instance segmentation.
• To overcome the limitations of manual annotation and improve the accuracy of zero-shot leaf segmentation models.
• To provide a user-friendly graphical user interface (GUI) for the entire data generation pipeline.

Main Methods:

• Proposed a novel method combining an off-the-shelf zero-shot segmentation model with structure-aware image generation.
• Utilized L-system growth rules to represent plant structural patterns for realistic image synthesis.
• Developed a GUI front-end to integrate the automatic dataset generation pipeline for enhanced usability.

Main Results:

• Successfully generated arbitrary numbers of instance mask and photorealistic plant image pairs without manual annotation.
• Achieved superior leaf instance segmentation performance compared to state-of-the-art zero-shot models on multiple plant species.
• Attained high AP@50 scores: 74.8 for Arabidopsis, 76.0 for Komatsuna, and 88.2 for Rhaphiloepsis.

Conclusions:

• The proposed automatic dataset generation method significantly enhances leaf instance segmentation accuracy.
• This approach eliminates the bottleneck of manual annotation, making advanced plant phenotyping more accessible.
• The integration of zero-shot models with structure-aware generation presents a powerful solution for plant image analysis.