Automated electronic medical record abstraction algorithm for radical prostatectomy outcomes

Variation in outcomes following radical prostatectomy and inclusion of prostate cancer surgery metrics in hospital ratings signal need for procedure-specific quality improvement (QI) efforts. We hypothesized that a novel electronic medical record (EMR)-based, automated algorithm-driven algorithm for surgical outcomes and quality metrics following robot-assisted laparoscopic radical prostatectomy (RALP) would demonstrate >90% sensitivity and specificity and significant inter-rater reliability (IRR) with National Surgical Quality Improvement Program (NSQIP) abstraction.

We developed an algorithm to automatically abstract RALP outcomes and quality metrics retrospectively from the EMR. Pathology results were abstracted through text extraction; surgical outcomes were abstracted using ICD-10 codes, CPT codes, and EMR data variables. Sensitivity, specificity, and IRR between the algorithm and NSQIP-abstraction were assessed using Cohen's kappa with statistical significance set P < 0.05.

Total of 927 cases were mutually tracked. IRR was highest for mortality (k = 1.00) and lowest for dialysis and ureteral obstruction (k = 0.00). IRR was fair for: sepsis (k = 0.28), renal insufficiency (k = 0.32), and prolonged NGT/NPO (k = 0.39); moderate: UTI (k = 0.50) and stage (k = 0.53); substantial: surgical margins (k = 0.94), urine leak (k = 0.60), C-Diff (k = 0.67), pneumonia (k = 0.80). Sensitivity of the algorithm was > 90% for all mutually tracked outcomes except rectal injury (0%) and specificity was >97%.

This novel algorithm for RALP outcomes matches or exceeds sensitivity and specificity of institutional NSQIP abstraction for all but 1 variable. Substantial agreement between the algorithm and NSQIP supports automated extraction of outcome metrics as an acceptable replacement for trained abstractors, and broader application provides opportunities to facilitate and reduce cost of outcomes and quality metric benchmarking.