Measurement specificity with modern methods: Using dimensions, facets, and items from personality assessments to...

Area of Science:

• Organizational Psychology
• Industrial-Organizational Psychology
• Psychometric Measurement

Background:

• Predicting work-related outcomes using personality measures is a long-standing research interest.
• Optimal operationalization level within personality hierarchies for predictive algorithms remains debated.

Purpose of the Study:

• To investigate the optimal level of personality operationalization (dimensions, facets, items) for developing predictive algorithms using machine learning (ML).
• To compare different scoring methods, content relevance, and sample sizes for criterion-related validity.

Main Methods:

• Employed machine learning (ML) techniques, including elastic net regression, to analyze personality data from 1,043 police officers.
• Assessed criterion-related validity based on personality operationalization level, scoring method, content relevance, and sample size.

Main Results:

• Empirically derived scores significantly outperformed unit weighting across all personality levels.
• Elastic net regression at the item level yielded the highest validity estimates.
• Using job-relevant items did not outperform using all item content in exploratory modeling.

Conclusions:

• Personality scoring should utilize narrow operationalizations, at least down to the facet level, for enhanced predictive validity.
• Machine learning (ML) offers a powerful tool for maximizing criterion-related validity and addressing theoretical questions in organizational science.