Automated ICD coding using extreme multi-label long text transformer-based models.

Area of Science:

• Natural Language Processing
• Medical Informatics
• Machine Learning

Background:

• Transformer models show promise for natural language processing tasks.
• Automated International Classification of Diseases (ICD) coding presents challenges due to extreme label sets and long text.
• Existing Transformer models like PLM-ICD and XR-Transformer are being explored for ICD coding.

Purpose of the Study:

• To investigate and optimize Transformer-based models for automated ICD coding.
• To address the challenges of extreme label sets and long text classification in ICD coding.
• To propose and evaluate a novel Transformer-based model, XR-LAT.

Main Methods:

• Optimized the state-of-the-art PLM-ICD model by training with longer sequence lengths.
• Extended the XR-Transformer model to support longer sequences for ICD coding tasks.
• Developed and trained a novel model, XR-LAT, utilizing a hierarchical code tree, label-wise attention, knowledge transferring, and dynamic negative sampling.

Main Results:

• Optimized PLM-ICD models achieved new state-of-the-art micro-F1 scores of 60.8% on MIMIC-III and 50.9% on MIMIC-II.
• XR-Transformer did not perform optimally across all metrics for ICD coding.
• XR-LAT models showed competitive performance, improving macro-AUC by 2.1% (MIMIC-III) and 5.1% (MIMIC-II) compared to previous state-of-the-art.

Conclusions:

• Optimized PLM-ICD models represent the new state-of-the-art for automated ICD coding on MIMIC-III and MIMIC-II datasets.
• The novel XR-LAT model offers competitive performance, demonstrating potential for complex ICD coding tasks.
• Transformer-based approaches are effective for addressing challenges in automated ICD coding.