Related Experiment Video
Updated: Aug 8, 2026
12:18
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach
Timo Seitz1, Franziska M Leipold1, Julius Protte1
1University of Mannheim, Mannheim, Germany.
Acta Psychologica
|August 6, 2026
Summary
This study introduces a hierarchical multinomial processing tree (MPT) model to accurately measure knowledge in multiple-choice tests, accounting for guessing strategies and improving scoring accuracy.
Area of Science:
- Psychometrics
- Educational Measurement
- Cognitive Psychology
Background:
- Multiple-choice tests can yield correct answers through guessing, not just knowledge.
- Traditional scoring methods are biased against specific response strategies, especially with a "Don't know" option.
- Accurate measurement of latent ability is crucial in educational and psychological assessments.
Purpose of the Study:
- To propose a novel hierarchical multinomial processing tree (MPT) model for improved scoring of multiple-choice tests.
- To account for individual differences in guessing tendencies and option preferences.
- To provide a more accurate measure of latent ability compared to conventional techniques.
Main Methods:
- Development of a hierarchical multinomial processing tree (MPT) model with crossed-random person and item effects.
- Estimation of the model using a Bayesian framework.
- Application to an empirical dataset from a general knowledge test with a "Don't know" option.
Main Results:
- The proposed hierarchical MPT model demonstrated good fit to the empirical data.
- Significant variability in model parameters was observed across individuals and items.
- The model showed higher convergent validities, indicating improved knowledge measurement.
Conclusions:
- Hierarchical MPT modeling offers a robust approach to psychometric questions, particularly for tests with guessing.
- The model provides a more nuanced understanding of response processes beyond simple correct/incorrect scoring.
- This methodology enhances the precision and validity of ability estimation in assessments.
Related Concept Videos
01:19Survival Tree
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a survival tree begins...
Building a Survival Tree
Constructing a survival tree begins...
01:13Multiple Comparison Tests
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:23McNemar's Test
McNemar's Test is a nonparametric statistical test used to determine if there is a significant difference in proportions between two related groups when the outcome is binary (e.g., yes/no, success/failure). It is beneficial when we have paired data, such as pre-test/post-test designs, where the same subjects are measured under two different conditions. The test is named after the statistician Quinn McNemar, who introduced it in 1947. It is commonly used in situations where subjects are...
01:09Decision Making: P-value Method
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
01:25Multiple Regression
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
00:55Ordinal Level of Measurement
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks in the...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks in the...