Enhancing data quality in medical concept normalization through large language models.

Area of Science:

• Natural Language Processing
• Medical Informatics
• Machine Learning

Background:

• Medical Concept Normalization (MCN) is essential for machine learning in healthcare.
• Existing MCN research often neglects data quality's impact.
• This study addresses MCN performance by focusing on data quality enhancement.

Purpose of the Study:

• To evaluate MCN performance under varying data quality conditions.
• To investigate methods for improving data quality using large language models (LLMs).
• To enhance MCN performance through data-driven quality improvements.

Main Methods:

• Conducted a data quality evaluation for MCN datasets.
• Employed ChatGPT for zero-shot and few-shot data augmentation.
• Assessed augmented data quality (correctness, comprehensiveness).
• Analyzed the impact of data quality on MCN model performance through experiments.

Main Results:

• Dataset duplication can skew MCN evaluation results.
• LLM-based augmentation (zero-shot, few-shot) may introduce data duplication.
• Careful design of augmentation strategies is needed to mitigate duplication.
• Including augmented data in test sets is vital for accurate evaluation.

Conclusions:

• Large language models (LLMs) can generate high-quality data for MCN.
• Few-shot learning with context-rich prompts and representative data is effective.
• The developed framework offers insights for data augmentation in deep learning.