Biomedical text normalization through generative modeling

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Artificial Intelligence

Background:

• Electronic health records (EHRs) contain vast amounts of unstructured text data.
• Inconsistent formatting of EHR text hinders its use in predictive modeling and clinical decision support.
• Large language models (LLMs) offer potential for standardizing medical text due to their contextual understanding.

Purpose of the Study:

• To develop and evaluate clinical text normalization pipelines using LLMs.
• To assess the effectiveness of various LLM-based normalization strategies.
• To compare LLM performance against traditional methods for medical text standardization.

Main Methods:

• Implemented four LLM-based normalization strategies: Zero-Shot Recall, Prompt Recall, Semantic Search, and Retrieval-Augmented Generation normalization (RAGnorm).
• Included a baseline using TF-IDF based String Matching.
• Evaluated performance on three SNOMED-mapped condition term datasets (oncology, institutional sample, common codes) and TAC 2017 drug labels (MedDRA terms).
• Measured performance using mean shortest path length and micro F1 score.

Main Results:

• RAGnorm demonstrated superior performance across all evaluated datasets.
• RAGnorm achieved a mean shortest path length of 0.21 on the oncology dataset.
• Achieved a micro F1 score of 88.01 on TAC 2017 task 4, outperforming other models.

Conclusions:

• Retrieval-focused LLM approaches, like RAGnorm, overcome limitations of traditional LLMs for text normalization.
• RAGnorm and similar retrieval techniques show significant promise for normalizing biomedical free text.
• Further exploration of these methods is recommended for advancing clinical data utilization.