Handling missing data in variational autoencoder based item response theory

Area of Science:

• Psychometrics
• Machine Learning
• Statistical Modeling

Background:

• High-dimensional Item Response Theory (IRT) models are crucial for educational and psychological assessments.
• Traditional IRT estimation methods struggle with large datasets and missing data.
• Variational Autoencoders (VAEs) show promise for efficient estimation but lack inherent missing data handling.

Purpose of the Study:

• To adapt and propose VAE-based methods for estimating high-dimensional IRT models with missing data.
• To compare the performance of these VAE methods against each other and traditional marginal maximum likelihood (MML) estimation.
• To evaluate the impact of increasing missing data levels on VAE method performance.

Main Methods:

• Adaptation of three existing VAE imputation techniques for the IRT context.
• Development of a novel VAE-based method for handling missing data in IRT.
• Simulation studies with varying dimensions (3D, 10D) and missing data proportions.
• Application of VAE models to a real-world algebra test dataset.

Main Results:

• VAE-based methods provide a time-efficient alternative to MML for IRT estimation.
• Performance of VAE methods is comparable to MML, especially with careful parameter tuning.
• Increased importance-weighted samples are necessary for VAE methods when missing data proportions are substantial.
• Demonstrated practical utility on an algebra test dataset.

Conclusions:

• VAE-based approaches offer a viable and efficient solution for estimating high-dimensional IRT models, particularly in the presence of missing data.
• The choice of VAE method and the number of samples are critical for optimal performance with extensive missingness.
• Further research into VAEs for psychometric modeling is warranted.