Hierarchical label-wise attention transformer model for explainable ICD coding

Area of Science:

• Medical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Accurate International Classification of Diseases (ICD) coding is crucial for classifying morbidity and mortality data.
• Existing methods for ICD code prediction from clinical documents require improvement in accuracy and explainability.

Purpose of the Study:

• To propose a hierarchical label-wise attention Transformer model (HiLAT) for explainable ICD code prediction.
• To evaluate the performance of HiLAT, particularly when combined with a continually pre-trained Transformer model (ClinicalplusXLNet).

Main Methods:

• Fine-tuning a pretrained Transformer model to represent clinical document tokens.
• Employing a two-level hierarchical label-wise attention mechanism to create label-specific document representations.
• Utilizing a feed-forward neural network for ICD code prediction based on document representations.

Main Results:

• HiLAT + ClinicalplusXLNet demonstrated superior F1 scores compared to state-of-the-art models for frequent ICD-9 codes in the MIMIC-III database.
• Attention weight visualizations provided insights into the model's prediction process, suggesting potential for explainability.
• The developed ClinicalplusXLNet model, based on XLNet-Base, showed effectiveness through continual pretraining on clinical notes.

Conclusions:

• The proposed HiLAT model offers a promising approach for accurate and explainable ICD code prediction.
• Combining HiLAT with advanced Transformer architectures like ClinicalplusXLNet significantly enhances prediction performance.
• The attention mechanism in HiLAT serves as a valuable tool for validating the clinical relevance of predicted ICD codes.