Machine learning methods applied to audit of surgical margins after curative surgery for head and neck cancer

Most surgical specialties have attempted to address concerns about the unfair comparison of outcomes by 'risk-adjusting' data to benchmark specialty-specific outcomes that are indicative of quality of care. We explore the ability to predict for positive margin status so that effective benchmarking that will account for complexity of case mix is possible. A dataset of care episodes recorded as a clinical audit of margin status after surgery for head and neck squamous cell carcinoma (n=1316) was analysed within the Waikato Environment for Knowledge Analyisis (WEKA) machine learning programme. The outcome was a classification model that can predict for positivity of tumour margins (defined as less than 1mm) using data on preoperative demographics, operations, functional status, and tumour stage. Positive resection margins of less than 1mm were common, and varied considerably between treatment units (19%-29%). Four algorithms were compared to attempt to risk-adjust for case complexity. The 'champion' model was a Naïve Bayes classifier (AUROC 0.72) that suggested acceptable discrimination. Calibration was good (Hosmer-Lemershow goodness-of-fit test p=0.9). Adjusted positive margin rates are presented on a funnel plot. Subspecialty groups within oral and maxillofacial surgery are seeking metrics that will allow for meaningful comparison of the quality of care delivered by surgical units in the UK. To enable metrics to be effective, we argue that they can be modelled so that meaningful benchmarking, which takes account of variation in complexity of patient need or care, is possible.