A Systematic Review of the Quality and Utility of Observer-Based Instruments for Assessing Medical Professionalism

Area of Science:

• Medical Education
• Healthcare Professionalism Assessment
• Psychometric Evaluation

Background:

• Professionalism is a critical competency in medical practice, covering behavioral and ethical domains.
• Existing observer-based instruments for assessing medical professionalism have limited documented psychometric properties and utility.

Purpose of the Study:

• To systematically review the psychometric properties and utility of observer-based instruments used for assessing professionalism in medical trainees.

Main Methods:

• A systematic review was conducted using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria for methodological quality.
• Ten instruments were identified and assessed for psychometric properties and utility, considering factors like scoring criteria and validity evidence.

Main Results:

• Sixteen studies evaluated ten instruments; most had acceptable methodological quality.
• The Education Outcomes Service (EOS) group questionnaire and Professionalism Mini-Evaluation Exercise (P-MEX) demonstrated the best psychometric properties.
• P-MEX exhibited superior utility compared to the EOS group questionnaire.

Conclusions:

• Two instruments, EOS and P-MEX, possess the best psychometric properties for assessing medical trainee professionalism.
• P-MEX is a viable option for program directors for formative assessment.
• Further research is recommended to gather more validity evidence for both instruments, especially for content validity in diverse cultural settings and for summative assessments.