Related Experiment Video
Updated: Jul 3, 2025

Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
Published on: December 11, 2016
ChatGPT for generating multiple-choice questions: Evidence on the use of artificial intelligence in automatic item
Yavuz Selim Kıyak1,2, Özlem Coşkun3, Işıl İrem Budakoğlu3
1Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara, Turkey. yskiyak@gazi.edu.tr.
Purpose:
Artificial intelligence, specifically large language models such as ChatGPT, offers valuable potential benefits in question (item) writing. This study aimed to determine the feasibility of generating case-based multiple-choice questions using ChatGPT in terms of item difficulty and discrimination levels.
Methods:
This study involved 99 fourth-year medical students who participated in a rational pharmacotherapy clerkship carried out based-on the WHO 6-Step Model. In response to a prompt that we provided, ChatGPT generated ten case-based multiple-choice questions on hypertension. Following an expert panel, two of these multiple-choice questions were incorporated into a medical school exam without making any changes in the questions. Based on the administration of the test, we evaluated their psychometric properties, including item difficulty, item discrimination (point-biserial correlation), and functionality of the options.
Results:
Both questions exhibited acceptable levels of point-biserial correlation, which is higher than the threshold of 0.30 (0.41 and 0.39). However, one question had three non-functional options (options chosen by fewer than 5% of the exam participants) while the other question had none.
Conclusions:
The findings showed that the questions can effectively differentiate between students who perform at high and low levels, which also point out the potential of ChatGPT as an artificial intelligence tool in test development. Future studies may use the prompt to generate items in order for enhancing the external validity of the results by gathering data from diverse institutions and settings.
More Related Videos
12:18A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
16:23Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
Published on: February 26, 2014
Related Concept Videos
Structure-Activity Relationships and Drug Design
SAR studies the intricate relationship between a drug's chemical structure and biological activity. It focuses on understanding how modifications to a drug's structure can influence...
Rational Dosage Regimen: Maintenance Dose and Loading Dose
In most cases, drugs are administered repetitively or infused continuously to maintain a steady-state concentration in the body. At a steady...
Analysis of Population Pharmacokinetic Data
Drug Administration and Therapy Phases: Overview
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...
Drug Discovery: Overview
Mechanistic Models: Overview of Compartment Models