DiffPlantCT: a training-free, annotation-free approach to cross-species plant CT image segmentation

Traditional deep learning-based plant computed tomography (CT) image segmentation methods require a large amount of high-quality manually labeled data for model training specific to each species, leading to substantial labor costs and poor adaptability to new species. These limitations hinder the application of CT imaging in large-scale cross-species plant phenotyping analysis. Therefore, developing annotation-free and training-free plant CT image segmentation methods is of significant research and application value in reducing research costs and promoting the efficiency of cross-species analysis. To achieve this, we introduce an unsupervised zero-shot segmentation framework for cross-species plant CT images, DiffPlantCT. It is a 2D-to-3D framework that first segments all 2D slices and then assembles them in their original order to generate a 3D CT segmentation. For each slice, this framework directly constructs discriminative clustering features by combining the general semantic priors provided by the self-attention layers in a pre-trained stable diffusion model with the intrinsic grayscale distribution of original image, thereby completely avoiding the need for manual annotations. The method ultimately outputs segmentation results solely through unsupervised clustering, achieving zero-shot generalization without any model training or fine-tuning. To evaluate the feasibility of DiffPlantCT in cross-species segmentation, we benchmark the segmentation performance on two public datasets (walnut fruit and barley spike) and two self-collected datasets (wheat spike and rice panicle). The results show that DiffPlantCT achieved the best performance, with a 41.6% improvement in overall mIoU compared to the state-of-the-art unsupervised method. For the first time, we demonstrate annotation-free, training-free segmentation of cross-species plant CT images successfully.