Related Experiment Video
Updated: Jul 27, 2025
05:21
Computerized Adaptive Testing System of Functional Assessment of Stroke
Published on: January 7, 2019
5.8K
The Impact of Item Model Parameter Variations on Person Parameter Estimation in Computerized Adaptive Testing With
1Department of Human Development and Quantitative Methodology, University of Maryland, College Park, MD, USA.
Applied Psychological Measurement
|June 7, 2023
Summary
Ignoring sibling item variations in psychometric testing has minimal impact on score accuracy but can introduce bias. Increasing test length can compensate for larger variations, with similar effects in linear and adaptive testing.
Area of Science:
- Psychometrics
- Educational Measurement
- Item Response Theory
Background:
- Sibling items from automatic generation share similar psychometric properties but have variations.
- Ignoring these variations simplifies computation but may affect scoring accuracy.
- Understanding the impact of within-family variance is crucial for accurate person parameter estimation.
Purpose of the Study:
- To explore the impact of item model parameter variations (within-family variance) on person parameter estimation.
- To investigate if test length can compensate for larger within-model variance.
- To determine if item pool characteristics influence the scoring impact of within-family variance.
- To compare these effects in linear versus Computerized Adaptive Testing (CAT).
Main Methods:
- Data generation using a related sibling model, assuming an identical sibling model for scoring.
- Manipulation of factors including test length, within-model variation size, and item model pool characteristics.
- Analysis of standard error, correlation between true and estimated scores, Root Mean Square Error (RMSE), and bias.
Main Results:
- Increased within-family variance did not significantly affect the standard error of scores.
- Test length compensated for larger within-model variance regarding score correlation and RMSE.
- Bias in scores was observed, tending towards the center, and was not compensated by test length.
- Computerized Adaptive Testing (CAT) showed similar results to linear tests but with higher efficiency.
Conclusions:
- While standard errors are stable, ignoring within-family variance can lead to biased ability estimates.
- Balanced item model pools are recommended to mitigate bias by canceling out 'fake-easy' and 'fake-difficult' item effects.
- Test length is a viable strategy to manage the impact of within-family variance on score accuracy.
Related Concept Videos
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
67
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
67
01:14Parametric Survival Analysis: Weibull and Exponential Methods
492
Parametric survival analysis models survival data by assuming a specific probability distribution for the time until an event occurs. The Weibull and exponential distributions are two of the most commonly used methods in this context, due to their versatility and relatively straightforward application.
Weibull Distribution
The Weibull distribution is a flexible model used in parametric survival analysis. It can handle both increasing and decreasing hazard rates, depending on its shape parameter...
Weibull Distribution
The Weibull distribution is a flexible model used in parametric survival analysis. It can handle both increasing and decreasing hazard rates, depending on its shape parameter...
492
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
85
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
85
01:26Modeling in Therapy
128
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
128
01:06Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
100
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
100
01:16Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
164
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
164