Comparative Analysis of ChatGPT-4 for Automated Mapping of Local Medical Terminologies to SNOMED CT
View abstract on PubMed
Summary
This summary is machine-generated.Retrieval-Augmented Generation (RAG) best maps local medical terms to SNOMED CT, improving healthcare data standardization. This AI approach shows promise for automated medical coding and Named Entity Recognition (NER) in clinical settings.
Area Of Science
- Medical Informatics
- Artificial Intelligence in Healthcare
- Clinical Terminology Standardization
Background
- Standardizing medical terminology is crucial for healthcare informatics, enhancing data interoperability and patient management.
- Accurate mapping of local medical terms to standardized vocabularies like SNOMED CT is essential for seamless data exchange.
- Existing methods for medical terminology mapping face challenges in accuracy and efficiency.
Purpose Of The Study
- To evaluate and compare four GPT-4-based approaches for mapping local medical terminologies to SNOMED CT.
- To assess the accuracy and error rates of baseline, prompt-engineered, fine-tuned, and Retrieval-Augmented Generation (RAG) models.
- To identify the most effective AI strategy for automated medical terminology mapping in a clinical context.
Main Methods
- Utilized 1,200 diagnostic terms from a Korean hospital for evaluation.
- Implemented and compared four GPT-4-based models: baseline, prompt-engineered, fine-tuned, and RAG.
- Assessed performance based on valid SNOMED CT term match rate, exact match rate, and structural error rate.
Main Results
- The RAG model demonstrated superior performance with a 96.2% valid SNOMED CT term match rate and a 57.6% exact match rate.
- The RAG model achieved the lowest structural error rate at 14%, outperforming other models.
- Fine-tuned and RAG models showed limitations in specificity but offered significant improvements over baseline and prompt-engineered methods.
Conclusions
- Retrieval-Augmented Generation (RAG) is a highly effective approach for mapping local medical terms to SNOMED CT, enhancing healthcare data standardization.
- AI-driven methods, particularly RAG, show significant potential for improving automated medical coding and Named Entity Recognition (NER) tasks.
- Further research is necessary to refine model specificity and validate these AI systems for widespread clinical adoption.