Related Experiment Video
Updated: Jul 11, 2025
06:48
Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.2K
Adjusted Residuals for Evaluating Conditional Independence in IRT Models for Multistage Adaptive Testing.
Peter W van Rijn1, Usama S Ali2,3, Hyo Jeong Shin4
1ETS Global, Amsterdam, The Netherlands. pvanrijn@etsglobal.org.
Psychometrika
|November 6, 2023
Summary
Multistage adaptive testing (MST) data violate item response theory (IRT) assumptions due to routing. Adjusted residuals are needed for accurate statistical inference in MST, as shown by PISA data analysis.
Area of Science:
- Statistics
- Psychometrics
- Educational Measurement
Background:
- Item response theory (IRT) models assume conditional independence of item responses given latent ability.
- Multistage adaptive testing (MST) designs involve routing decisions that can violate this core IRT assumption.
- This violation introduces dependencies in the data, impacting statistical inference.
Purpose of the Study:
- To investigate the impact of routing in MST on the conditional independence assumption of IRT models.
- To evaluate the appropriateness of generalized residuals for analyzing MST data.
- To propose and validate adjustments to statistical methods for MST data.
Main Methods:
- Examined the relationship between MST routing and data patterns using concepts from log-linear models.
- Assessed the suitability of generalized residuals for item pair frequencies in IRT.
- Developed and tested adjusted residuals tailored to specific MST designs through simulation and real data analysis.
Main Results:
- Standard generalized residuals are inappropriate for MST data without modifications.
- Adjustments to residuals are necessary and depend on the complexity of the MST routing.
- The adjusted residuals demonstrated satisfactory Type I error rates in simulations and real data applications.
Conclusions:
- The conditional independence assumption in IRT is challenged by MST routing.
- Adjusted residuals are crucial for valid statistical inference in MST.
- Findings have implications for interpreting results from large-scale assessments like PISA.
Related Concept Videos
01:21Introduction to Test of Independence
2.3K
In statistics, the term independence means that one can directly obtain the probability of any event involving both variables by multiplying their individual probabilities. Tests of independence are chi-square tests involving the use of a contingency table of observed (data) values.
The test statistic for a test of independence is similar to that of a goodness-of-fit test:
The test statistic for a test of independence is similar to that of a goodness-of-fit test:
2.3K
01:15Assumptions of Survival Analysis
136
Survival models analyze the time until one or more events occur, such as death in biological organisms or failure in mechanical systems. These models are widely used across fields like medicine, biology, engineering, and public health to study time-to-event phenomena. To ensure accurate results, survival analysis relies on key assumptions and careful study design.
136
01:21Friedman Two-way Analysis of Variance by Ranks
207
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
207
01:20Comparing the Survival Analysis of Two or More Groups
200
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
200
01:19Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K
01:16Hypothesis Test for Test of Independence
3.6K
The test of independence is a chi-square-based test used to determine whether two variables or factors are independent or dependent. This hypothesis test is used to examine the independence of the variables. One can construct two qualitative survey questions or experiments based on the variables in a contingency table. The goal is to see if the two variables are unrelated (independent) or related (dependent). The null and alternative hypotheses for this test are:
H0: The two variables (factors)...
H0: The two variables (factors)...
3.6K