Assessing ovarian cancer via laparoscopic video: inter-rater and intra-rater reliability

Ovarian Cancer is often diagnosed at an advanced stage. Central to treatment is a correct initial assessment and the probability of complete resection of visible cancer. The predictive index value (PIV) is a validated tool for laparoscopic procedures, assessing the chance of complete resection by evaluating 7 intra-abdominal areas. The final PIV determines the type of treatment to which women with ovarian cancer are referred. Gynecologic oncologists perform the procedure and assessment, and agreement and correct staging are essential. This study examines the agreement among gynecologists when assessing videos using the PIV.

Laparoscopic videos were collected from October 2021 to January 2024, comprising 20 videos, of which 5 were duplicated. Each video was edited by a gynecologic oncologist, representing the seven intra-abdominal areas according to the PIV model. Eight gynecologic oncologists, eight gynecologists specialized in benign conditions, and eight residents in gynecology were invited to assess the videos using the PIV. The ratings from the participants were examined according to laparoscopic real-time scores. Inter-rater and intra-rater agreement were quantified using kappa statistics.

Twenty-one individuals participated in the study. Correct assessment of intra-abdominal areas, that is, agreement with laparoscopic real-time scores, ranged from 58.4% to 81.7%. The probability of correct scores among groups of gynecologists ranged from 34% (95% CI 14.9 to 60.2) to 98.9% (95% CI 95.9 to 99.7), with the highest scores among gynecologic oncologists. The inter-rater agreement of all participants ranged from moderate to substantial agreement (Light κ = 0.436-0.624). For specific intra-abdominal sites, it ranged from fair to perfect agreement (Light κ = 0.181-0.829).

Assessing ovarian cancer based on videoclips was challenging, even for specialists in gynecologic oncology, indicating that edited videos reduce accuracy in evaluating disease burden. Gynecologic oncologists achieved the highest scores among all groups, suggesting that specialized training and knowledge improve consistency and that the assessment should remain an expert task.