Related Experiment Video
Updated: May 14, 2025
07:13
A Two-interval Forced-choice Task for Multisensory Comparisons
Published on: November 9, 2018
10.8K
Item Discrimination, Difficulty, and Response Time for 4-Option Multiple Choice Questions Versus 3-Option Multiple
Min Jiang1, Jacqueline E McLaughlin1
1University of North Carolina (UNC), UNC Eshleman School of Pharmacy, Chapel Hill, NC, USA.
American Journal of Pharmaceutical Education
|April 25, 2025
Summary
Reducing multiple-choice questions (MCQs) from four to three options improved item discrimination and decreased response time without affecting item difficulty in biostatistics exams. This suggests three-option MCQs enhance assessment efficiency in pharmacy education.
Area of Science:
- Educational Assessment
- Psychometrics
- Pharmacy Education
Background:
- Standardized testing in pharmacy education often utilizes multiple-choice questions (MCQs).
- The number of options in MCQs (e.g., 3 vs. 4) can influence psychometric properties and test-taking time.
- Optimizing MCQ format is crucial for efficient and effective student evaluation.
Purpose of the Study:
- To compare item difficulty, discrimination, and response time between 3-option and 4-option MCQs.
- To evaluate the impact of reducing distractors on MCQ performance metrics.
- To inform best practices for MCQ design in biostatistics and pharmacy education.
Main Methods:
- Twenty biostatistics exam items were analyzed, administered as 4-option MCQs (2017-2020) and 3-option MCQs (2021-2024).
- 3-option items were created by removing the least preferred distractors from original 4-option items.
- Paired t-tests compared mean item difficulty, discrimination, and response time using Examsoft data.
Main Results:
- Item difficulty remained comparable: 89.75% ± 8.45% for 4-option vs. 86.45% ± 15.98% for 3-option (p =.08).
- Item discrimination significantly improved for 3-option items (0.29 ± 0.09) compared to 4-option items (0.23 ± 0.11, p =.03).
- Mean response time decreased for 3-option MCQs (38.10 ± 17.78 s) versus 4-option MCQs (42.50 ± 14.70 s, p =.02).
Conclusions:
- Three-option MCQs offer potential advantages in pharmacy education by enhancing discrimination and reducing exam time.
- Reducing distractors to create 3-option MCQs maintains item difficulty while improving psychometric quality.
- Further research into optimizing MCQ design is recommended to balance assessment efficiency and psychometric integrity.
More Related Videos
07:34Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
17.3K
06:43The Crossmodal Congruency Task as a Means to Obtain an Objective Behavioral Measure in the Rubber Hand Illusion Paradigm
Published on: July 26, 2013
15.9K
Related Concept Videos
01:13Multiple Comparison Tests
3.8K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.8K
01:23McNemar's Test
90
McNemar's Test is a nonparametric statistical test used to determine if there is a significant difference in proportions between two related groups when the outcome is binary (e.g., yes/no, success/failure). It is beneficial when we have paired data, such as pre-test/post-test designs, where the same subjects are measured under two different conditions. The test is named after the statistician Quinn McNemar, who introduced it in 1947. It is commonly used in situations where subjects are...
90
01:21Friedman Two-way Analysis of Variance by Ranks
108
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
108
01:17Cochran's Q Test
148
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
148
02:01Factorial Design
12.7K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
12.7K
02:01Group Design
8.8K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.8K