Related Experiment Video
Updated: Jun 12, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Understanding Transformer-Based Classifications of Medical Text Using a Large Language Model for the Attribution of
Fangwen Zhou1, Ashirbani Saha2, Muhammad Afzal3
1Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4L8, Canada, 1 905-525-9140 ext 22208.
Generative large language models like GPT-4o struggle as standalone explainers for biomedical text classification. Traditional methods like SHAP and integrated gradients (IG) offer more reliable and efficient explanations for model interpretability.
Area of Science:
- Biomedical Informatics
- Artificial Intelligence
- Natural Language Processing
Background:
- Deep learning models, particularly transformer architectures, excel in biomedical literature classification but lack interpretability.
- Explainable AI (XAI) methods like SHAP and integrated gradients (IG) improve transparency but are computationally intensive.
- Generative large language models (LLMs) present a potential new avenue for creating interpretable, context-aware explanations.
Purpose of the Study:
- To evaluate GPT-4o as a standalone, end-to-end perturbation-based explainer for BioLinkBERT text classification.
- To compare GPT-4o's explanation faithfulness and semantic alignment against established SHAP and IG baselines.
- To assess the computational efficiency and cost-effectiveness of GPT-4o compared to traditional XAI methods.
Main Methods:
- A fine-tuned BioLinkBERT model classified 200 studies from McMaster PLUS and Clinical Hedges for methodological rigor.
- Stratified sampling over-represented low-confidence predictions to rigorously test explainers.
- GPT-4o, SHAP, and IG generated token-level feature attributions, with GPT-4o using iterative masking under two prompting schemes.
- Explanation quality was assessed using modified area over the perturbation curve (AOPC) and correlation analyses.
Main Results:
- SHAP (AOPC 0.222) and IG (AOPC 0.225) provided consistent and faithful explanations, identifying key tokens related to study rigor.
- GPT-4o exhibited significantly lower faithfulness (AOPC 0.025-0.029) and produced divergent attributions.
- Correlation analysis showed moderate alignment between SHAP and IG (r=0.367), but limited correlation with GPT-4o (r≤0.032).
- GPT-4o was computationally intensive and costly, while IG was the most time-efficient.
Conclusions:
- Current generative LLMs are limited as standalone perturbation explainers for biomedical text classification.
- GPT-4o struggles with accurate feature importance synthesis via iterative masking, lacking the reliability of traditional XAI frameworks.
- Future research should explore specialized prompt engineering, whole-word strategies, and hybrid approaches for LLM-based explanations.