Can I retake it? Exploring subgroup differences and criterion-related validity in promotion retesting

Area of Science:

• Industrial-Organizational Psychology
• Human Resources Management
• Psychometrics

Background:

• Job applicant retesting is a growing practice with unknown effects.
• Key evaluation factors for staffing include subgroup differences and criterion-related validity.
• Understanding retest impacts is crucial for fair and effective hiring.

Purpose of the Study:

• To investigate how retesting affects subgroup differences in test scores.
• To determine the impact of retesting on the criterion-related validity of job knowledge tests.
• To analyze retest effects within a sample of internal candidates seeking promotion.

Main Methods:

• Examined retest effects in 403 internal candidates for a promotion.
• Utilized job performance data for all candidates, irrespective of test outcome.
• Analyzed subgroup differences (gender, age, race) and validity coefficients.

Main Results:

• Retest improvements varied by subgroup: females and younger candidates improved more than males and older candidates.
• Black candidates showed less improvement compared to other racial groups.
• Retest scores were stronger predictors of job performance (r=.38) than initial scores (r=.27).

Conclusions:

• Retesting does not harm criterion-related validity and may enhance it.
• Retesting can reduce adverse impact for some subgroups (e.g., females) but increase it for others (e.g., older individuals).
• The practice of retesting requires careful consideration of its differential subgroup impacts.