Large-Scale Survey Data Analysis with Penalized Regression: A Monte Carlo Simulation on Missing Categorical...

Area of Science:

• Social Sciences
• Statistics
• Machine Learning

Background:

• The big data era necessitates advanced machine learning methods for analyzing complex datasets.
• Penalized regression offers a powerful approach for building interpretable prediction models.
• Handling missing categorical predictors in large-scale social science data remains a challenge.

Purpose of the Study:

• To investigate penalized regression for predictive modeling with missing categorical predictors in social science research.
• To emulate real-world social science large-scale data, including Likert-scaled, multiple-category, and count variables.
• To assess the utility of penalized regression methods, specifically group Mnet, for handling grouped data effects.

Main Methods:

• Monte Carlo simulation study to emulate large-scale social science data.
• Simulation of Likert-scaled, multiple-category, and count variables.
• Application of penalized regression methods, including group Mnet, to address categorical predictors and grouping effects.
• Validation using a real large-scale dataset and examination of variable selection counts.

Main Results:

• Penalized regression, particularly methods considering grouping effects, is viable for large-scale social science survey data.
• Variable selection counts are crucial for mitigating bias from data-splitting in model validation.
• Utilizing large-scale data effectively can help mitigate the impact of nonignorable missingness.

Conclusions:

• Penalized regression is a suitable method for analyzing large social science survey data, even with missing categorical predictors.
• The study highlights the importance of variable selection counts in model validation.
• Leveraging large datasets is a promising strategy for addressing missing data challenges in social science research.