Retest effects in medical selection interviews

Area of Science:

• Medical Education
• Psychometrics
• Applicant Assessment

Background:

• Repetition of cognitive tests improves scores, but interview retest effects are understudied.
• Interviews are crucial for selecting medical students and trainees, making retest effects a significant concern.
• Understanding retest effects is vital as applicants often reapply to programs.

Purpose of the Study:

• To determine if retest improvements occur in repeated interviews.
• To ascertain if improvements are due to general or specific interview experience.
• To assess the impact of repeat interviews on criterion validity.

Main Methods:

• Compared interview scores of applicants across different medical programs and years.
• Analyzed applicants interviewed by multiple programs versus those interviewed by the same program in consecutive years.
• Correlated initial and repeat interview scores with a social understanding test.

Main Results:

• General interview experience did not yield score improvements.
• Specific experience (same program, consecutive years) led to significant score increases.
• Criterion validity of interviews remained unaffected by repeat testing.

Conclusions:

• Re-interviewed unsuccessful medical school applicants at the same institution tend to score higher.
• This score increase is likely a retest effect, not a genuine improvement in ability.
• The study suggests interview retest effects do not compromise criterion validity.