Reproducibility of subtle MRI findings suggestive of minimal endometriosis: agreement, Kappa paradox, and
Hilton Leao Filho1, Marcelo Figueiredo Alves2, Mariana Sousa de Castro2
1Department of Radiology, Hospital do Coração, São Paulo, Brazil. hiltonmlf@gmail.com.
Purpose:
To evaluate intraobserver and interobserver agreement in the interpretation of subtle MRI findings classified as minimal endometriosis, to assess the impact of the Kappa paradox in low-prevalence compartments, and to compare blinded rereads with the original prospective reports.
Materials And Methods:
In this retrospective study, 100 pelvic MRI examinations (50 originally reported as minimal or incipient endometriosis and 50 originally reported as normal) were independently reviewed by three radiologists on two blinded occasions 4-8 weeks apart. Seven anatomical compartments were assessed. Agreement was measured with Cohen's Kappa, Fleiss' Kappa, PABAK, Gwet's AC1, and positive/negative agreement. Original reports were used as a clinical reference for descriptive comparison, and exact McNemar tests assessed directional asymmetry of discordance.
Results:
Intraobserver agreement for any finding was fair (mean Kappa = 0.36, 95%CI 0.16-0.54; AC1 = 0.49), and interobserver agreement was also fair (Fleiss Kappa = 0.28; AC1 = 0.40). Agreement was consistently stronger for negative than for positive classifications. Ovarian findings showed the highest reproducibility, whereas very low-prevalence compartments showed a marked Kappa paradox. Multicategory analysis of uterosacral ligament laterality reduced agreement, mainly because unilateral-versus-bilateral disagreement remained frequent among positive cases. Comparison with original reports showed slight-to-moderate agreement (Kappa 0.18-0.46), with specificity (76-86%) consistently exceeding sensitivity (32-66%), indicating a more restricted threshold on blinded rereads, particularly for uterosacral ligament involvement.
Conclusions:
Agreement for subtle MRI findings interpreted as minimal endometriosis was only fair and clearly lower than that usually reported for deep endometriosis. Low-prevalence compartments were strongly affected by the Kappa paradox, and the low-to-moderate discordance with original reports, driven primarily by lower sensitivity on blinded rereads, supports the need for clearer operational criteria and more standardized interpretation.
