Predicting and explaining with machine learning models: Social science as a touchstone.

Area of Science:

• Social Sciences
• Computational Social Science
• Predictive Modeling

Background:

• Machine learning (ML) models have achieved significant success in predictive tasks within the natural sciences.
• However, their application and benefits in the social sciences, particularly for predicting life trajectories, remain less evident and often unsuccessful by traditional metrics.

Purpose of the Study:

• To investigate the reasons behind the performance gap of ML models in social science prediction.
• To highlight prediction as a crucial goal in social science, alongside explanation.
• To propose a novel modeling approach to enhance ML model efficacy in social science.

Main Methods:

• Comparative analysis of two social science case studies against a natural science paradigm example.
• Identification of specific constraints hindering pure ML prediction in social science contexts.
• Development of an integrative modeling framework.

Main Results:

• The study identifies key constraints impeding the success of pure ML prediction in social science.
• It underscores the importance of prediction as a scientific goal in social sciences, comparable to explanation.
• The proposed integrative approach offers a potential solution to bridge the performance gap.

Conclusions:

• Integrating explanatory models with predictive ML models is essential for advancing social science research.
• This hybrid approach can overcome limitations of pure ML prediction in complex social systems.
• Future research should focus on implementing and validating such integrative strategies.