Explaining transformer-based classification of radiology reports

Area of Science:

• Artificial Intelligence in Radiology
• Machine Learning for Medical Imaging
• Explainable AI (XAI)

Background:

• Deep learning models for radiological report classification often lack transparency.
• Explainability is crucial for clinical adoption and trust in AI tools.
• Confounding data can impact the reliability of AI models in research.

Purpose of the Study:

• To validate and explain a pretrained deep learning classification model.
• To apply the model for removing confounding data from radiological datasets.
• To demonstrate the feasibility of explainable radiological report classification.

Main Methods:

• Two radiologists annotated 2038 MRI head reports for abnormality and small vessel disease.
• Pretrained transformer models were fine-tuned on 80% of the data and validated on the remaining 20%.
• SHapley Additive exPlanations (SHAP) were employed to interpret model predictions.

Main Results:

• The models achieved high classification performance (ROC AUC of 0.98 for abnormality, 0.99 for small vessel disease).
• SHAP analysis successfully highlighted key terms driving the model's classifications.
• The approach demonstrated effective identification and explanation of confounding data.

Conclusions:

• The validated pretrained transformer model effectively detects confounding data in radiological research cohorts.
• Explainable AI outputs facilitate expert review, error analysis, and iterative refinement of AI tools.
• This method supports the transparent and trustworthy integration of AI into clinical workflows.