A supervised learning approach to estimating IRT models in small samples | JoVE Visualize

Area of Science:

Psychometrics
Machine Learning
Educational Measurement

Background:

Item response theory (IRT) models are widely used in educational and psychological assessments.
Traditional IRT parameter estimation relies on the likelihood function, which can be unreliable in small samples, leading to biased estimates and large standard errors.
There is a need for robust IRT estimation methods that perform well with limited data.

Purpose of the Study:

To introduce a novel, likelihood-free approach for estimating item response theory (IRT) model parameters.
To develop and evaluate neural network (NN) based methods for small-sample IRT estimation.
To demonstrate the advantages of the NN approach over existing Bayesian estimation techniques.

Main Methods:

A novel estimation approach was developed that derives features from response data and maps them to item parameters using neural networks (NNs).
Three types of NNs were implemented to obtain both point estimates and confidence intervals for IRT parameters.
The proposed NN approach was evaluated using a simulation study, comparing its performance against Bayesian estimation with Markov chain Monte Carlo (MCMC) methods.

Main Results:

The NN-based approach demonstrated superior performance compared to Bayesian MCMC methods in terms of the quality of point estimates and confidence intervals.
The NN method was significantly faster than traditional Bayesian estimation techniques.
The simulation results confirmed the effectiveness of the NN approach for small-sample IRT parameter estimation.

Conclusions:

Neural networks provide a powerful and efficient alternative for item response theory (IRT) parameter estimation, especially in small sample sizes.
This likelihood-free NN approach facilitates real-time item pretesting and enhances the security of new item development in online testing environments.
The developed NN methods offer improved accuracy and speed, making them valuable for practical applications in psychometrics and educational measurement.