OSC-CO²: coattention and cosegmentation framework for plant state change with multiple features

Area of Science:

• Computer Vision
• Machine Learning
• Plant Science

Background:

• Traditional cosegmentation and coattention methods struggle with segmenting objects exhibiting morphological changes across different imaging modalities and views, particularly plants.
• Accurate segmentation of plant imagery is crucial for high-throughput phenotyping and understanding plant growth dynamics.

Purpose of the Study:

• To introduce the Object State Change using Coattention-Cosegmentation (OSC-CO^2) framework for accurate segmentation of plants with evolving morphology.
• To address the limitations of existing methods in handling multi-modal, multi-view plant imaging data.
• To present a new dataset, CosegPP+, for evaluating segmentation performance on plant growth sequences.

Main Methods:

• Developed an end-to-end unsupervised deep-learning framework (OSC-CO^2) that integrates coattention-based Convolutional Neural Networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs).
• The framework processes, analyzes, selects, and combines segmentation results to produce a final segmented image.
• Utilized plant growth sequences captured by infrared, visible, and fluorescence cameras on a remote sensing, high-throughput phenotyping platform.

Main Results:

• OSC-CO^2 demonstrated superior performance compared to state-of-the-art segmentation and cosegmentation methods.
• Segmentation accuracy was improved by a range of 3% to 45%.
• The CosegPP+ dataset provided quantitative validation of the framework's efficacy.

Conclusions:

• The OSC-CO^2 framework effectively addresses the challenge of segmenting objects with changing states, especially in complex plant imagery.
• This approach significantly enhances segmentation accuracy in high-dimensional, multi-modal, and multi-view plant imaging.
• The developed framework and dataset contribute to advancing automated plant phenotyping and analysis.