Related Experiment Video
Updated: Jan 17, 2026
07:08
Estimate the Cognitive Load Using Electrocardiographic Measure: A Human-AI Collaborative Task
Published on: December 5, 2025
190
Faculty versus artificial intelligence chatbot: a comparative analysis of multiple-choice question quality in
Anup Kumar D Dhanvijay1, Amita Kumari1, Mohammed Jaffer Pinjar1
1Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India.
Advances in Physiology Education
|September 22, 2025
Summary
Artificial intelligence (AI) chatbots can generate multiple-choice questions (MCQs) for medical education, but faculty-created questions remain superior. AI-generated MCQs showed comparable discrimination but were easier and had more ineffective distractors than faculty-authored Physiology MCQs.
Area of Science:
- Medical Education
- Artificial Intelligence in Education
- Physiology Assessment
Background:
- Multiple-choice questions (MCQs) are crucial for medical education assessment.
- Human-generated MCQs require significant time and pedagogical expertise.
- AI offers automated MCQ generation, but its educational validity needs evaluation.
Purpose of the Study:
- To compare the psychometric quality of Physiology MCQs generated by faculty versus an AI chatbot (DeepSeek R1).
- To assess difficulty index, discrimination index, and nonfunctional distractors for both question types.
Main Methods:
- 200 MCQs developed: 100 by faculty, 100 by AI.
- 50 MCQs from each group administered to undergraduate medical students.
- Item analysis using difficulty index (DIFI), discrimination index (DI), and nonfunctional distractors (NFDs).
Main Results:
- AI-generated MCQs had a significantly higher DIFI (0.64) than faculty MCQs (0.47).
- No significant difference in DI between AI and faculty MCQs (P = 0.17).
- Faculty MCQs had significantly fewer NFDs (median 0) than AI MCQs (median 1).
Conclusions:
- AI-generated MCQs have comparable discrimination but are easier and contain more ineffective distractors.
- AI shows promise for MCQ generation but requires refinement in distractor quality and item difficulty.
- AI can supplement, but not replace, human expertise in assessment design.
Related Concept Videos
01:16Non-equilibrium in the Cell
5.3K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
5.3K
01:13Multiple Comparison Tests
4.4K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.4K
01:16Bioequivalence: Overview
1.9K
Pharmaceutical equivalents, by definition, are drug products with the same active ingredient in the same quantities, encapsulated in identical dosage forms, and intended for the same administration routes. These pharmaceutical equivalents are deemed bioequivalent if the bioavailability of the active entity in the drug preparations is similar. Moreover, pharmaceutical equivalents demonstrating bioequivalence are also regarded as therapeutically equivalent. This means that when used as directed,...
1.9K
01:24Overview of Anatomy and Physiology
34.0K
Human anatomy is the scientific study of the body's structures. Some of these structures are very small and can only be observed and analyzed with the assistance of a microscope. Other larger structures can readily be seen, manipulated, measured, and weighed. The word "anatomy" comes from a Greek root that means "to cut apart." Human anatomy was first studied by observing the body's exterior and the wounds of soldiers and other injuries. Later, physicians were allowed to...
34.0K