Related Experiment Videos
Differential prediction and the use of multiple predictors: the omitted variables problem
Paul R Sackett1, Roxanne M Laczo, Zachary P Lippe
1Dept of Psychology, University of Minnesota, Elliott Hall, 75 East River Road, Minneapolis, MN 55455, USA. psackett@tc.umn.edu
The Journal of Applied Psychology
|December 4, 2003
Summary
Differential prediction analysis using moderated regression requires careful consideration of multiple predictors. Including a general factor significantly altered findings in many cases, highlighting the risk of omitted variables in individual predictor analyses.
Area of Science:
- Psychometrics
- Regression Analysis
- Differential Prediction
Background:
- Moderated regression is a common method for assessing differential prediction based on demographic factors like race or gender.
- Existing methodologies lack clear guidance on analyzing multiple predictors individually versus collectively within selection systems.
- Individual predictor analysis risks an omitted variable bias, potentially misrepresenting predictive accuracy.
Purpose of the Study:
- To investigate the impact of analyzing multiple predictors individually versus in combination for differential prediction.
- To assess the influence of including a general factor in differential prediction analyses.
- To examine predictive bias in the context of Army personnel selection.
Main Methods:
- Utilized Army Project A data, encompassing 79 predictor-criterion combinations involving personality measures.
- Performed moderated regression analyses to detect differential prediction by race.
- Compared results from individual predictor analyses with analyses including a general factor (Armed Services Vocational Aptitude Battery general factor).
Main Results:
- Traditional individual predictor analysis revealed predictive bias by intercept in 45 instances and by slope in 7 instances.
- Incorporating the Armed Services Vocational Aptitude Battery general factor as an additional predictor changed the bias conclusions in 32 cases for the intercept and 3 cases for the slope.
- The inclusion of the general factor substantially altered the detection of predictive bias.
Conclusions:
- Analyzing predictors individually in moderated regression can lead to omitted variable bias.
- The inclusion of a general factor is crucial for accurate differential prediction analysis in complex selection systems.
- Findings underscore the importance of considering predictor sets rather than isolated variables to avoid misinterpreting predictive bias.