Related Experiment Video
Updated: Jun 28, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Constructing synthetic datasets with generative artificial intelligence to train large language models to classify
Onkar Litake1, Brian H Park1, Jeffrey L Tully1
1Division of Perioperative Informatics, Department of Anesthesiology, University of California, San Diego, La Jolla, CA 92037, United States.
Objectives:
To compare performances of a classifier that leverages language models when trained on synthetic versus authentic clinical notes.
Materials And Methods:
A classifier using language models was developed to identify acute renal failure. Four types of training data were compared: (1) notes from MIMIC-III; and (2, 3, and 4) synthetic notes generated by ChatGPT of varied text lengths of 15 (GPT-15 sentences), 30 (GPT-30 sentences), and 45 (GPT-45 sentences) sentences, respectively. The area under the receiver operating characteristics curve (AUC) was calculated from a test set from MIMIC-III.
Results:
With RoBERTa, the AUCs were 0.84, 0.80, 0.84, and 0.76 for the MIMIC-III, GPT-15, GPT-30- and GPT-45 sentences training sets, respectively.
Discussion:
Training language models to detect acute renal failure from clinical notes resulted in similar performances when using synthetic versus authentic training data.
Conclusion:
The use of training data derived from protected health information may not be needed.
Related Concept Videos
01:27Dialysis
Acute kidney injury develops suddenly and can be caused by pre-renal causes (e.g., hypovolemia, shock), intrinsic renal causes (e.g., acute tubular necrosis), or post-renal causes (e.g., urinary obstruction). In contrast, chronic renal failure progresses gradually over time and is often...
01:16Non-equilibrium in the Cell