Proficiency in identifying, managing and communicating medical errors: feasibility and validity study assessing two...

Area of Science:

• Medical Education
• Healthcare Professionalism
• Clinical Skills Assessment

Background:

• Evaluating communication skills and professionalism in graduate medical education presents significant challenges.
• A novel instrument was developed to address the need for reliable assessment of these critical competencies.

Purpose of the Study:

• To develop, test, and validate a new instrument for evaluating communication skills and professionalism.
• To assess the feasibility, acceptability, and reliability of an Objective Standardized Clinical Examination (OSCE) for competency evaluation.

Main Methods:

• An OSCE was designed using a medication error scenario.
• The assessment instrument focused on distinct domains: context, communication, error management, empathy, electronic medical record (EMR) and electronic medical information resource (EMIR) use, and global rating.
• Faculty and standardized patients (SPs) evaluated 56 trainees.

Main Results:

• Substantial inter-rater reliability was found between faculty (Fleiss k=0.71) and excellent intraclass correlation (ICC=0.80).
• Agreement between faculty and SPs was lower (Fleiss k=0.36), with good overall instrument conformity (ICC=0.74).
• 75% of trainees performed satisfactorily or higher across all domains; 60% requested EMR use training due to lack of feedback.

Conclusions:

• An OSCE utilizing a medical error scenario is a viable instrument for assessing professionalism, communication, EMR use, and medical error management.
• The findings support the use of OSCEs for comprehensive competency assessment in graduate medical education.