Related Experiment Video
Updated: Aug 4, 2026
05:21
Computerized Adaptive Testing System of Functional Assessment of Stroke
Published on: January 7, 2019
A comparison of the difficulty, reliability and validity of complex multiple choice, multiple response and multiple
Annual Conference on Research in Medical Education. Conference on Research in Medical Education
|January 1, 1977
Summary
Multiple true-false (MTF) items are easier, more reliable, and more valid than complex multiple choice (CMC) items in health sciences education. Test developers should consider alternative item formats to CMC for improved assessment quality.
Area of Science:
- Educational assessment
- Health professions education
Background:
- Complex multiple choice (CMC) items are widely used in health sciences education.
- The psychometric properties of CMC items compared to other formats are not fully understood.
Purpose of the Study:
- To compare the psychometric properties of complex multiple choice (CMC) items with multiple response (MR) and multiple true-false (MTF) items.
- To evaluate item difficulty, reliability, and validity across different item formats.
Main Methods:
- A comparative study design was employed.
- Psychometric analyses were conducted on CMC, MR, and MTF item formats.
Main Results:
- Multiple true-false (MTF) items demonstrated greater ease of use.
- MTF items exhibited higher reliability and validity compared to CMC items.
- The findings suggest potential limitations in the psychometric performance of CMC items.
Conclusions:
- The study recommends that test developers, including organizations like the National Board of Medical Examiners, explore alternative item formats.
- Consideration of MTF items may enhance the quality and effectiveness of health sciences assessments.
Related Concept Videos
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:13Multiple Comparison Tests
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:29Measures of Intelligence
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
01:22Self-Report Tests of Personality
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
01:17Cochran's Q Test
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square distribution,...
01:29Theory of Attribution II: Kelley's Covariation Theory
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus: Comparing...