Development and initial validation of a program director's evaluation form for medical school graduates

Area of Science:

• Medical Education
• Graduate Medical Training
• Program Evaluation

Background:

• Developed an initial program director evaluation form in the early 1990s at Uniformed Services University (USU).
• Revised the evaluation form to align with Accreditation Council for Graduate Medical Education core competencies.
• Incorporated items reflecting USU's unique military context.

Purpose of the Study:

• To gather evidence on the feasibility, reliability, and validity of the revised program director evaluation survey.
• To assess the psychometric properties of a new evaluation tool for postgraduate year 1 (PGY-1) medical trainees.

Main Methods:

• Collected PGY-1 data from program directors overseeing military medical trainees (USU graduates 2010-2011).
• Utilized exploratory factor analysis (EFA) to determine factorial validity.
• Assessed internal consistency reliability and correlated ratings with medical school GPA and USMLE Step exam performance.

Main Results:

• Exploratory factor analysis identified five key factors: Medical Expertise, Military-unique Practice, Professionalism, System-based Practice, and Communication and Interpersonal Skills.
• The revised form demonstrated good reliability and feasibility.
• Factors correlated more strongly with initial clerkship GPA and USMLE Step 3 performance.

Conclusions:

• The revised program director evaluation form is a valid and reliable instrument.
• The tool effectively measures first-year internship performance in medical graduates.
• The findings support the use of this revised evaluation tool in graduate medical education.