Interobserver Agreement Between Artificial Intelligence, Radiologist, and Gynecologist in Hysterosalpingography...

Background: Infertility is a common reproductive health disorder that affects roughly 10-15% of couples during their reproductive period. Hysterosalpingography (HSG) is a widely utilized imaging modality for assessing uterine cavity morphology and fallopian tube patency and continues to play a central role in infertility investigations. Nevertheless, the interpretation of HSG findings may vary according to the experience and expertise of the evaluator, potentially leading to inconsistencies in clinical decision-making. Although artificial intelligence (AI) has demonstrated considerable potential in medical image analysis across various specialties, evidence regarding its application in the interpretation of HSG examinations remains scarce. Therefore, this study aimed to evaluate the level of agreement among radiologists, gynecologists, and an AI-based system in the assessment of identical HSG images. Methods: In this retrospective study, a total of 1443 HSG images obtained from 414 women who underwent hysterosalpingography as part of an infertility evaluation between January 2021 and January 2025 were reviewed. Cases with incomplete clinical records or suboptimal image quality were excluded from the analysis. All examinations were independently assessed by an experienced radiologist, a gynecologist specializing in infertility management, and a multimodal artificial intelligence system based on ChatGPT-5, with each evaluator blinded to the assessments of the others and to the patients' clinical information. Image interpretation included the evaluation of contrast distribution, peritoneal spill, uterine cavity findings, tubal patency, and overall HSG impression, which were categorized according to predefined diagnostic criteria. The primary outcome was the degree of interobserver agreement among the evaluators. Agreement analyses were performed using Cohen's kappa (κ) and Gwet's AC1 coefficients. Analyses were conducted using IBM SPSS Statistics (version 30.0; IBM Corp., Armonk, NY, USA) and R statistical software (version 4.4.0; R Foundation for Statistical Computing, Vienna, Austria). Statistical significance was set at p < 0.05 (two-sided). Results: A total of 1443 HSG images obtained from 414 women were included in the final analysis. The mean age of the study population was 30.97 ± 5.59 years, and primary infertility accounted for 87.9% of cases. The average number of images acquired per examination was 3.49 ± 1.05. According to Cohen's kappa analysis, the highest levels of agreement were observed for the assessment of image artifacts and contrast medium distribution. Agreement between the AI system and the radiologist was particularly strong for contrast medium distribution (κ = 0.757). For the overall interpretation of HSG findings, AI demonstrated substantial agreement with the radiologist (κ = 0.637), exceeding the level of agreement observed between the radiologist and the gynecologist (κ = 0.363). In contrast, concordance involving AI was lower for the evaluation of uterine abnormalities, intrauterine filling defects, and tubal patency. When agreement was reassessed using Gwet's AC1 statistic, concordance coefficients were consistently higher than the corresponding kappa values across all evaluator pairs. Near-perfect agreement between AI and the radiologist was identified for contrast medium distribution (AC1 = 0.954), peritoneal spill (AC1 = 0.893), and patterns of peritoneal contrast passage (AC1 = 0.841). Procedures performed under local anesthesia yielded a significantly greater number of images than those conducted under general anesthesia (3.86 ± 0.86 vs. 3.08 ± 1.10, p < 0.001). No significant associations were detected between abnormal HSG findings and either infertility type or anesthetic technique. In multivariable analysis, the use of general anesthesia was independently associated with a lower image count, whereas the presence of tubal pathology emerged as an independent predictor of acquiring a greater number of images during the examination. Conclusions: Our findings indicate that AI-assisted interpretation of HSG images has the potential to complement expert assessment, showing substantial concordance in several key diagnostic domains. While the technology appears promising as a decision-support tool in infertility evaluation, further research and refinement are warranted, particularly regarding the assessment of tubal and uterine pathologies.