Real-Life Inter-Rater Variability of the PRAETORIAN Score Values

(1) Background: The PRAETORIAN score is a tool developed for postoperative evaluation of the position of subcutaneous implantable cardioverter-defibrillator systems. The aim of our study was to evaluate the real-life inter-rater variability of the PRAETORIAN score, based on chest radiographs of S-ICD patients reviewed by independent clinical raters. (2) Methods: Postoperative chest X-rays of patients that underwent S-ICD implantation were evaluated by five clinical raters who gave values of the PRAETORIAN score. Ratings were then compared in a fully crossed manner to determine the inter-rater variability of the attributed scores. (3) Results: In total, 87 patients were included in the study. In the case of the most important final risk category of the PRAETORIAN score, the mean Light's kappa was 0.804, the Fleiss' kappa was 0.249, and the intraclass correlation was 0.38. The final risk category was identically determined by all five raters in 75.86% of patients, by four raters in 14.94%, and by three raters in 9.20% of patients. (4) Conclusions: The overall inter-rater variability of the PRAETORIAN score in a group of electrophysiologists experienced in S-ICD implantation, yet previously naive to the PRAETORIAN score, and self-trained in its utilization, was only modest in our study. Appropriate use of the score might require training of clinical raters.