Self-supervised feature extraction from image time series in plant phenotyping using triplet networks

Area of Science:

• Plant biology
• Computational biology
• Machine learning

Background:

• Image-based profiling is crucial for high-throughput plant phenotyping and herbicide discovery.
• Extracting meaningful features from unlabeled plant images presents a significant challenge.

Purpose of the Study:

• To develop a novel, data-driven, self-supervised approach for extracting meaningful feature representations from plant time-series images.
• To improve the efficiency and accuracy of plant phenotyping for applications like herbicide discovery.

Main Methods:

• Utilized a self-supervised, data-driven method using time as a proxy for image similarity.
• Employed an ImageNet-pretrained architecture as a base feature extractor, extended with a triplet network.
• Refined and reduced feature dimensionality by ranking similarities between time points.

Main Results:

• Generated compact and organized representations of plant phenotypes without requiring labels.
• Demonstrated superior performance in clustering, image retrieval, and classification tasks.
• Showcased the versatility of the approach for phenotypic profiling.

Conclusions:

• The developed self-supervised method offers a powerful and versatile tool for plant phenotyping.
• This approach accelerates the discovery of novel herbicides by improving feature extraction from plant images.
• The methodology can be adapted using other phenotype similarity measures.