Intra/inter-observer reliability of cystoscopic sphincter evaluation in men undergoing sling surgery

To assess the intra/inter-observer reliability of cystoscopic sphincter evaluation (CSE) in men undergoing sling surgery for urinary incontinence and if possible to evaluate its correlation with the final clinical decision.

Two expert urologists prospectively filmed and recorded, incontinent patient's cystoscopies according to a standard scenario. Anonymised recordings where randomly offered to the same observer twice. The observers (medical students, urology residents and full urologist with 0-5, 5-10, >10 years of practice, respectively) were asked to assess and score the recordings without knowing any of the patients' characteristics.

In total, 37 recordings were scored twice by the 26 observers. The intraclass correlation coefficient (ICC) for intra-observer reliability of the CSE was 0.54 (moderate), 0.58 (moderate) and 0.60 (substantial) for medical students, residents, and urologists, respectively. However, when stratifying observers according to their experience, the lowest agreement values were found between experts with >10 years of experience. The inter-observer reliability for the CSE ICCs ranged between 0.31and 0.53, with the lowest ICC value observed between urologists (0.31).

The study demonstrates poor intra- and inter-observer reliability of the CSE. According to these results, a CSE does not add valuable information to the clinical evaluation. In this scenario, it should not be considered in isolation from the patient's characteristics.