Comparative testing of reliability and audit utility of ordinal objective calculus complexity scores. Can we make an...

To assess the reliability of the Guy's Stone Score, the Seoul National University Renal Stone Complexity (S-ReSC) score and the S.T.O.N.E. scores in percutaneous nephrolithotomy (PCNL), and assess their utility in discriminating outcomes [stone free rate (SFR), complications, need for multiple PCNL sessions, and auxiliary procedures] valid across parameters of experience of surgeon, independence from surgical approach, and variations in institution-specific instrumentation.

A prospectively maintained database of two tertiary institutions was analysed (606 cases). Institutes differed in instrumentation, while the overall surgical team comprised: two trainees (experience <100 cases), two junior consultants (experience 100-200 cases), and two senior surgeons (experience >1000 cases). Scores were assigned and re-assigned after 4 months by one trainee and an expert surgeon. Inter-rater and test-retest agreement were analysed by Cohen's κ and intraclass correlation coefficient. Multivariate logistic regression models were created adjusting outcomes for the institution, comorbidity, Amplatz size, access tract location, the number of punctures, the experience level of the surgeon, and individual scoring system, and receiver operating curves were analysed for comparison.

Despite some areas of inconsistencies, individually all scores had excellent inter-rater and test-retest concordance. On multivariable analyses, while the experience of the surgeon and surgical approach characteristics (such as access tract location, Amplatz size, and number of punctures) remained independently associated with different outcomes in varying combinations, calculus complexity scores were found consistently to be independently associated with all outcomes. The S-ReSC score had a superior association with SFR, the need for multiple PCNL sessions, and auxiliary procedures.

Individually all scoring systems performed well. On cross comparison, the S-ReSC score consistently emerged to be more superiorly associated with all outcomes, signifying the importance of the distributional complexity of the calculus (which also indirectly amalgamates the influence of stone number, size, and anatomical location) in discriminating outcomes. Our study proves the utility of scoring systems in prognosticating multiple outcomes and also clarifies important aspects of their practical application including future roles such as benchmarking, audit, training, and objective assessment of surgical technique modifications.