Related Experiment Video
Updated: Jun 28, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative
Annessa Kernberg1, Jeffrey A Gold1, Vishnu Mohan1
1Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Sciences University, Portland, OR, United States.
ChatGPT-4 generates clinical notes with significant errors, primarily omissions, and variable quality. Its accuracy decreases with longer transcripts, making it unsuitable for current clinical documentation.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
Background:
- Accurate medical documentation is vital for patient care and communication.
- Inaccuracies and documentation burden contribute to physician burnout.
- Current AI solutions for documentation have limitations in accuracy and workflow integration.
Purpose of the Study:
- To evaluate the accuracy and quality of Subjective, Objective, Assessment, and Plan (SOAP) notes generated by ChatGPT-4.
- To compare AI-generated notes against gold-standard transcripts of patient encounters.
- To identify and categorize errors in AI-generated medical documentation.
Main Methods:
- Simulated patient-provider encounters were transcribed.
- ChatGPT-4 generated SOAP notes from these transcripts.
- Notes were compared to gold standards, with errors categorized (omissions, additions, incorrect information) and quality assessed using the Physician Documentation Quality Instrument (PDQI).
Main Results:
- ChatGPT-4 produced an average of 23.6 errors per case, with 86% being omissions.
- Note accuracy varied significantly between replicates (52.9% correct across all 3).
- Accuracy was inversely correlated with transcript length and data complexity.
Conclusions:
- ChatGPT-4 exhibits substantial variability in error rates, accuracy, and note quality.
- The model's performance is compromised by transcript length and data complexity.
- Current AI-generated notes do not meet clinical standards, necessitating caution before widespread adoption.
More Related Videos
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
06:32Author Spotlight: Automated Deep Brain Stimulation for Parkinson's Disease - Exploring the Possibilities and Challenges of Home Monitoring
Published on: July 14, 2023
Related Concept Videos
Methods of Documentation II: POMR
Methods of Documentation IV: Focus Charting
It typically involves three columns for recording information:
Methods of Documentation III: PIE
Methods of Documentation I: Source-Oriented Records
In an SOR, each discipline involved in patient care maintains a separate medical record section. This record-keeping method enables easy tracking of patient progress and ensures healthcare staff have access to up-to-date information.
Key Attributes include the following:
Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
Purpose of Health Records I
Here's a breakdown of how health records serve these purposes: