Multi-label text classification via secondary use of large clinical real-world data sets

Area of Science:

• Medical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Procedural coding is a significant administrative burden for clinicians.
• Natural Language Processing (NLP) offers potential solutions for automating coding tasks.
• Developing efficient tools can alleviate clinician workload and improve healthcare efficiency.

Purpose of the Study:

• To create an application that predicts medical procedure codes by analyzing clinical operative notes.
• To streamline clinician workflow and enhance administrative efficiency through automated coding assistance.
• To evaluate the performance of various machine learning models for procedure code prediction.

Main Methods:

• Utilized a dataset of approximately 350,000 German surgery notes for model adaptation.
• Modeled procedure code prediction as a multi-label classification task.
• Compared transformer-based models (medBERT.de, surgeryBERT.at) with non-contextual models (fastText, CNN, SVM, Logistic Regression).

Main Results:

• Support Vector Machines (SVM) achieved a mean average precision of 0.872 for reports longer than 512 sequences, outperforming BERT-based models.
• BERT models (medBERT.de, surgeryBERT.at) and fastText achieved competitive performance, with mean average precisions of 0.880 and 0.867 respectively.
• FastText demonstrated similar performance to BERT models with lower hardware requirements and better prediction explainability.

Conclusions:

• Predicting procedure codes from operative reports is feasible as a multi-label classification task.
• Classical machine learning methods like SVM can be highly effective for this task.
• FastText offers a computationally efficient alternative to BERT-based models for procedure code prediction and interpretability.