Development and temporal validation of a proof-of-concept artificial intelligence model for binary classification of...

To develop and temporally validate a proof-of-concept deep learning model applied to digitized ThinPrep cervical cytology and human papillomavirus (HPV) cotest results for cervical dysplasia identification and triage support. This retrospective single-center study included women undergoing cervical cotesting at Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy, between January and September 2023 (development cohort January-July; temporal validation cohort August-September). The model distinguished true-negative cotest cases from HPV-positive low-grade/high-grade squamous intraepithelial lesions; atypical squamous cells of undetermined significance or cannot exclude high-grade lesion, glandular abnormalities, and invasive lesions were excluded. Two convolutional neural network architectures (ResNet50 and DenseNet201) were trained using transfer learning and evaluated by three-fold cross-validation. The best-performing model was validated on an independent temporal cohort (732 image patches; n = 63 women). ThinPrep slides from 337 women generated 3867 image patches for model development. All performance metrics are reported at the image-patch level and not at the patient level. The ResNet50 ensemble achieved the highest patch-level performance (area under the receiver operating characteristic curve: 96%) in temporal validation. At the optimized 30% threshold, sensitivity was 93%, negative predictive value 94%, and accuracy 89%. Reference cotest categories were significantly associated with colposcopic impression (P < 0.001) and histological outcome (P < 0.001 in the development cohort). The model demonstrated high patch-level discriminative ability for the selected binary cotest classification. Although these findings suggest a potential role for artificial intelligence-assisted cytology within screening programs, prospective multicenter studies with patient-level analysis are required before clinical implementation.