Using a standardised patient assessment to measure professional attributes

Area of Science:

• Medical Education
• Professionalism Assessment
• Clinical Skills Evaluation

Background:

• Professionalism is crucial in medical education, yet assessment methods are underdeveloped.
• Existing research primarily defines professionalism and teaches interpersonal behaviors, neglecting robust assessment strategies.

Purpose of the Study:

• To evaluate the reliability and validity of assessing doctor professionalism using standardized patients (SPs).
• To determine if specific professional attributes can be reliably measured within the ECFMG clinical skills assessment (CSA).

Main Methods:

• Utilized data from over 35,000 candidates and 370,000 SP encounters in the ECFMG CSA.
• Analyzed professionalism-related component scores, focusing on interpersonal skills like rapport, attentiveness, attitude, and empathy.

Main Results:

• Reliability scores for professionalism components ranged from 0.61 to 0.70.
• Younger doctors and recent graduates received higher ratings.
• Female candidates outperformed male candidates across all measured professional dimensions.
• Professional qualities showed a weak correlation with basic science and clinical proficiency.

Conclusions:

• Standardized patient examinations can reliably and validly measure certain aspects of professional behavior.
• Alternative methods like surveys and critical incident techniques may be better suited for other professional attributes.