Reproducibility of digital pathology features extracted from deep learning and foundational AI models on sequential...

Numerous deep learning (DL) and foundational models (FMs) designed for digital pathology analysis employ advanced feature extraction techniques from digitized hematoxylin and eosin (H&E) slides obtained from tissue sections. These extracted features may serve as the input for complex models enabling automated classification tasks, slide searching, and other various medical inferences. Subtle variations in morphology observed in distinct cross sections of the same tissue sample may introduce variability that may be of little to no clinical relevance. The reproducibility of features derived from DL models and FMs on ostensibly identical tissue cross sections remains largely unexplored. In this study, we aimed to characterize the reproducibility of features extracted from three sequential cross sections of 50 independent normal prostate samples. Slides were sectioned at approximately 50 μm intervals and we extracted cell-type specific counts, proportions, and spatial features per slide using DL models, along with feature embeddings from popular digital pathology FMs. Reproducibility of the features across sequential cross sections of tissues was assessed using intra-class correlation coefficient (ICC) for DL model features, while extracted FM embedding distributions were evaluated via maximum mean discrepancy (MMD), and Wasserstein's distance. The median [IQR] ICC estimates for cell-type specific proportion and spatial features from the cross sections was 0.838 [0.764, 0.897], and 0.873 [0.809, 0.923] respectively. The bootstrapped mean differences of ICC estimates for the features were significantly lower (p-value < 0.05) for distant cross sections compared to the two neighboring cross sections. The feature embeddings derived from state-of-the-art digital pathology FMs showed an overall high agreement with the median [IQR] MMD of 0.06 [0.016, 0.169], and median [IQR] Wasserstein's distance of 0.021 [0.011, 0.038]. These findings demonstrate that the features derived from multiple cross sections of the same tissue exhibit overall high reproducibility, while also illustrating the expected increase in intra-sample variability as a function of physical distance between sections.