IRTCI: Item Response Theory for Categorical Imputation

Area of Science:

• Data Science
• Statistics
• Machine Learning

Background:

• Missing data values limit statistical analyses and model building.
• Existing imputation methods have varying impacts on downstream applications.
• Categorical data imputation is a significant challenge in data preprocessing.

Purpose of the Study:

• To introduce a novel categorical imputation method based on Item Response Theory (IRT).
• To compare the performance of the new IRT-based method against current machine learning imputation techniques.
• To evaluate imputation accuracy and predictive performance across diverse datasets and missing data scenarios.

Main Methods:

• Developed Item Response Theory for Categorical Imputation (IRTCI).
• Compared IRTCI with k-nearest neighbors (kNN), multiple imputed chained equations (MICE), and Datawig.
• Tested methods on ordinal, nominal, and binary datasets with varying missing data proportions and patterns.

Main Results:

• IRTCI demonstrated competitive performance against established imputation methods.
• The novel IRTCI approach outperformed several existing methods under various conditions.
• Probabilistic category assignment in IRTCI contributes to its robust performance.

Conclusions:

• IRTCI provides a theoretically grounded and effective approach for categorical data imputation.
• The method offers a promising alternative for handling missing values in diverse datasets.
• IRTCI's performance suggests its utility in improving statistical inference and model building.