Data Quality Estimation Via Model Performance: Machine Learning as a Validation Tool.

Area of Science:

• Neurosurgery
• Medical Informatics
• Machine Learning

Background:

• Classifying neurosurgical operative reports is crucial for data analysis.
• Previous expert-derived classification methods achieved suboptimal F-scores (≤0.74).
• The need for improved accuracy in automated medical text classification is evident.

Purpose of the Study:

• To enhance short text classification of neurosurgical operative reports using deep learning.
• To investigate the impact of refining the target variable on classifier performance.
• To apply deep learning models to real-world neurosurgical data.

Main Methods:

• Redesigning the target variable based on pathology, localization, and manipulation type.
• Implementing deep learning models for text classification.
• Evaluating model performance using accuracy and F1-score on a real-world dataset.

Main Results:

• The redesigned target variable significantly improved deep learning model performance.
• Achieved a classification accuracy of 0.995 and an F1-score of 0.990.
• Successfully classified operative reports into 13 distinct classes.

Conclusions:

• Improving target variable definition is key to effective machine learning-based text classification.
• Deep learning models, when guided by well-defined targets, can achieve high accuracy in medical report analysis.
• Machine learning can serve as a tool to validate human-generated codifications.