Quality indicators for cancer pain management: natural language processing and machine learning from electronic...

The continual evaluation of quality indicators (QIs) for cancer pain management is essential to maintaining and improving its quality. However, free-text notes in electronic medical records (EMRs) must be reviewed to accurately assess pain management QIs, and the heavy workload of this process can be limiting.Therefore, this study evaluated the performance of natural language processing and machine learning for assessing pain management QIs using EMR data.

This single-centre cross-sectional study included adult patients with cancer who died at a Japanese university hospital between 1 January 2022 and 31 December 2024. Clinical notes concerning inpatients and outpatients were extracted from the EMR system. A model was developed to automatically identify documentation related to pain management from free-text notes. We then compared the model's QIs assessment with a manual review concerning the number of patients who underwent pain screening.

The study included 865 patients, and 2 119 377 clinical records were used to evaluate QIs. The model achieved 85%-96% accuracy, and the F1 score ranged from 0.38 to 0.58 for identifying pain-screening documentation. The pain screening rate for the centre's inpatients was 100%, according to the manual and model-based evaluations. For the outpatients, the rates were 35.0% and 35.9%, respectively, by the manual and model-based evaluations.

Pain management QIs can be accurately assessed using natural language processing and machine learning applied to EMRs, with performance comparable to that of manual reviews. This approach may improve cancer pain management by providing clinician feedback and visualising achievement rates.