Standardized Video Interview Scores Correlate Poorly with Faculty and Patient Ratings

Area of Science:

• Medical Education
• Residency Admissions
• Professional Competency Assessment

Background:

• The Association of American Medical Colleges developed the Standardized Video Interview (SVI) to evaluate key applicant competencies.
• The SVI assesses professionalism, communication, and interpersonal skills crucial for residency.
• The correlation between SVI scores and other established competency measures remains unexamined.

Purpose of the Study:

• To investigate the relationship between SVI scores and faculty evaluations of emergency medicine (EM) applicants.
• To determine if SVI scores correlate with patient-reported communication and care ratings.
• To assess the SVI's validity as a predictor of essential residency applicant skills.

Main Methods:

• Retrospective analysis of prospectively collected data from fourth-year EM clerkship students applying to residency.
• Collected faculty ratings on professionalism and patient care/communication.
• Utilized the Communication Assessment Tool (CAT) for patient-reported outcomes.
• Compared SVI scores with faculty and CAT scores using Spearman's rank correlation.

Main Results:

• Thirty-six of 43 EM clerkship students provided SVI scores; 32 had available CAT scores.
• No significant correlation was found between SVI scores and faculty ratings of professionalism (rho = 0.09, p = 0.13).
• No significant correlation was observed between SVI scores and faculty assessments of patient care/communication (rho = 0.12, p = 0.04).
• SVI scores showed no significant correlation with patient-reported CAT scores (rho = 0.11, p = 0.06).

Conclusions:

• Current findings suggest SVI scores do not correlate with faculty or patient assessments in EM applicants.
• Further validation studies are required to establish the SVI's predictive utility for residency competencies.
• The SVI's role in assessing professionalism and communication skills needs additional investigation.