Related Experiment Video
Updated: Apr 5, 2026
07:01
Working Memory Training for Older Participants: A Control Group Training Regimen and Initial Intellectual Functioning Assessment
Published on: September 20, 2020
5.4K
Does Educator Training or Experience Affect the Quality of Multiple-Choice Questions?
Emily M Webb1, Jonathan S Phuong1, David M Naeger1
1Department of Radiology and Biomedical Imaging, University of California, San Francisco, 505 Parnassus Avenue, M-391, San Francisco, CA 94143-0628.
Academic Radiology
|August 17, 2015
Summary
Physicians trained in writing multiple-choice questions (MCQs) produce significantly fewer flawed questions. Educator experience alone did not impact question quality, with untrained faculty, residents, and students performing similarly.
Area of Science:
- Medical Education
- Assessment and Evaluation
Background:
- Physicians often lack formal training in writing effective multiple-choice questions (MCQs).
- Well-constructed MCQs are crucial for accurately assessing medical knowledge and ensuring educational objectives are met.
- Flawed MCQs can lead to misinterpretation and inaccurate assessment of learner competency.
Purpose of the Study:
- To investigate the impact of formal training in MCQ writing on the quality of questions produced by medical educators.
- To determine if educator experience level influences the prevalence of flawed MCQs.
- To compare the error rates in MCQs written by faculty with and without MCQ writing training, as well as by residents and medical students.
Main Methods:
- A total of 200 unedited MCQs from a senior medical student radiology elective question bank were analyzed.
- Questions were categorized by author: faculty with prior MCQ writing training, other faculty, residents, and medical students (50 questions each).
- Two independent readers assessed each question for 11 predefined flaws based on established literature.
Main Results:
- MCQs written by faculty with prior MCQ writing training demonstrated significantly fewer errors (mean 0.4 errors/question) compared to other groups (mean 1.5-1.7 errors/question; P < .001).
- No significant differences in error rates were found between untrained faculty, residents, and medical students (P values .35-.91).
- Trained faculty had a lower percentage of flawed questions (34%) compared to other faculty (76%), residents (74%), and students (88%).
Conclusions:
- Formal training in effective MCQ writing significantly reduces the incidence of flawed questions among medical faculty.
- Educator experience alone does not correlate with improved MCQ quality; untrained faculty, residents, and students exhibit similar error rates.
- Implementing structured training programs for MCQ development is essential for enhancing the quality of medical education assessments.
Related Concept Videos
01:20Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
2.8K
Evaluation of the teaching process enables the nurse to determine if the patient's learning needs were met and if training was effective. If the expected outcomes are not met, the care plan is revised, and additional education or reinforcement is provided. Nurses can ask questions after the session or obtain feedback to assess the patient's understanding of the topic.
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
2.8K
01:09Comparing Experimental Results: Student's t-Test
6.5K
The t-test is a statistical method used to compare the sample mean with a population mean or compare two means from two data sets. The test statistic is calculated from the standard deviation, mean, and number of measurements in the data set at a selected confidence interval and then compared to a table of critical values at this confidence level. If the test statistic is smaller than the critical value, the null hypothesis is accepted. In this case, we state that the difference between the...
6.5K
03:00Self-Evaluation: Self-Enhancement and Self-Verification
5.9K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.9K
01:17Cochran's Q Test
1.1K
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
1.1K
01:29Theory of Attribution II: Kelley's Covariation Theory
937
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus:...
937