Secondary Use of Clinical Problem List Entries for Neural Network-Based Disease Code Assignment

Area of Science:

• Medical Informatics
• Computational Linguistics
• Machine Learning

Background:

• Clinical information systems store vast amounts of electronic health record (EHR) data.
• This data, often semi-structured and partially annotated, is suitable for supervised, data-driven neural network analysis.
• Automated coding of clinical problem lists is crucial for data standardization and analysis.

Purpose of the Study:

• To explore automated coding of clinical problem list entries using the International Classification of Diseases (ICD-10).
• To evaluate the performance of different neural network architectures for this task.
• To identify limitations and potential improvements in automated medical coding.

Main Methods:

• Evaluation of three neural network architectures: fastText baseline, character-level Long Short-Term Memory (LSTM), and a downstreamed RoBERTa model with a custom language model.
• Testing was performed on 50-character clinical problem list entries against the top 100 ICD-10 three-digit codes.
• Analysis included neural network activation and investigation of false positives/negatives.

Main Results:

• The fastText baseline achieved a macro-averaged F1-score of 0.83.
• The character-level LSTM model reached a macro-averaged F1-score of 0.84.
• The RoBERTa model demonstrated superior performance with a macro-averaged F1-score of 0.88.

Conclusions:

• Advanced neural network models, particularly RoBERTa, show significant potential for automated clinical problem list coding.
• Inconsistent manual coding practices were identified as a primary factor limiting the accuracy of automated systems.
• Further improvements in data quality and standardization are necessary to optimize automated medical coding performance.