Natural language processing for automated surveillance of intraoperative neuromonitoring in spine surgery

We sought to develop natural language processing (NLP) methods for automated detection and characterization of neuromonitoring documentation from free-text operative reports in patients undergoing spine surgery. We included 13,718 patients who received spine surgery at two tertiary academic medical centers between December 2000 - December 2020. We first validated a rule-based NLP method for identifying operative reports containing neuromonitoring documentation, comparing performance to standard administrative codes. We then trained a deep learning model in a subset of 993 patients to characterize neuromonitoring documentation and identify events indicating change in status or difficulty establishing baseline signals. Performance of the deep learning model was compared to gold-standard manual chart review. In our patient population, 3,606 (26.3%) patients had neuromonitoring documentation identified using NLP. Our NLP method identified notes containing neuromonitoring documentation with an F1-score of 1.0, surpassing performance of standard administrative codes which had an F1-score of 0.64. In the subset of 993 patients used for training, validation, and testing a deep learning model, the prevalence of change in status was 6.5% and difficulty establishing neuromonitoring baseline signals was 6.6%. The deep learning model had an F1-score = 0.80 and AUC-ROC = 1.0 for identifying change in status, and an F1-score = 0.80 and AUC-ROC = 0.97 for identifying difficulty establishing baseline signals. Compared to gold standard manual chart review, our methodology has greater efficiency for identifying infrequent yet important types of neuromonitoring documentation. This method may facilitate large-scale quality improvement initiatives that require timely analysis of a large volume of EHRs.