Related Experiment Video
Updated: Jul 1, 2025

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
Published on: September 4, 2019
ChatGPT to generate clinical vignettes for teaching and multiple-choice questions for assessment: A randomized
Özlem Coşkun1, Yavuz Selim Kıyak1, Işıl İrem Budakoğlu1
1Department of Medical Education and Informatics, Gazi University, Ankara, Turkey.
Aim:
This study aimed to evaluate the real-life performance of clinical vignettes and multiple-choice questions generated by using ChatGPT.
Methods:
This was a randomized controlled study in an evidence-based medicine training program. We randomly assigned seventy-four medical students to two groups. The ChatGPT group received ill-defined cases generated by ChatGPT, while the control group received human-written cases. At the end of the training, they evaluated the cases by rating 10 statements using a Likert scale. They also answered 15 multiple-choice questions (MCQs) generated by ChatGPT. The case evaluations of the two groups were compared. Some psychometric characteristics (item difficulty and point-biserial correlations) of the test were also reported.
Results:
None of the scores in 10 statements regarding the cases showed a significant difference between the ChatGPT group and the control group (p > .05). In the test, only six MCQs had acceptable levels (higher than 0.30) of point-biserial correlation, and five items could be considered acceptable in classroom settings.
Conclusions:
The results showed that the quality of the vignettes are comparable to those created by human authors, and some multiple-questions have acceptable psychometric characteristics. ChatGPT has potential in generating clinical vignettes for teaching and MCQs for assessment in medical education.
Related Concept Videos
Randomized Experiments
Simple randomization
Simple...
Group Design
What is an Experiment?
Blinding
Crossover Experiments
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Controls in Experiments

