Evaluating GPT models for clinical note de-identification.

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Data Privacy

Background:

• Digitalization of healthcare necessitates secure clinical data management.
• Patient privacy is a critical concern in handling sensitive health information.
• Existing methods for data de-identification and synthetic data generation require optimization.

Purpose of the Study:

• To evaluate GPT-3.5 and GPT-4 for de-identifying clinical notes.
• To assess the performance of these models in generating synthetic clinical data.
• To optimize computational efficiency using API access and zero-shot prompt engineering.

Main Methods:

• Utilized GPT-3.5 and GPT-4 via API access.
• Employed zero-shot prompt engineering for de-identification and synthetic data generation.
• Evaluated model performance using precision, recall, F1 score, and accuracy metrics.

Main Results:

• GPT-4 demonstrated superior performance over GPT-3.5 in de-identifying clinical notes.
• GPT-4 achieved a precision of 0.9925, recall of 0.8318, F1 score of 0.8973, and accuracy of 0.9911.
• The study confirmed the potential of large language models for healthcare data privacy.

Conclusions:

• GPT-4 is a highly capable tool for enhancing patient privacy in clinical data.
• This research establishes a benchmark for balancing data utility and privacy in healthcare.
• The findings support the use of advanced AI for secure clinical data management and research enablement.