Sampling variability of the success ratio in predictor-based selection.

Area of Science:

• Psychometrics
• Statistical Modeling
• Predictive Analytics

Background:

• Predictor-based selection aims to maximize success ratios.
• The Taylor-Russell formula is a common but often inadequate estimation method.
• Sampling variability of the success ratio is poorly understood.

Purpose of the Study:

• To address deficiencies in estimating predictor-based selection success ratios.
• To derive the sampling distribution of the success ratio for key selection scenarios.
• To quantify the accuracy of success ratio predictions.

Main Methods:

• Analysis of three single-stage selection scenarios: restricted quota, threshold, and mixed quota/threshold.
• Derivation of the sampling distribution for the success rate statistic.
• Numerical evaluation of derived distributions.
• Example applications to demonstrate sampling variability.

Main Results:

• The Taylor-Russell formula's inadequacy is confirmed.
• Sampling distributions for success ratios were derived for common selection methods.
• Numerical evaluation methods for these distributions were discussed.
• Example applications showed substantial sampling variability in real-world predictor-based selections.

Conclusions:

• Existing methods for predicting selection success are often insufficient.
• Understanding the sampling variability of the success ratio is crucial for accurate prediction.
• The derived distributions provide a more robust framework for evaluating predictor-based selection strategies.