The OSSE: Development and validation of an "Objective Structured Supervision Evaluation"

Area of Science:

• Medical Education
• Patient Safety
• Physician Training

Background:

• Faculty physician competence in supervision is crucial for patient safety, yet lacks standardized training and assessment.
• Current evaluations like the Objective Structured Teaching Evaluation (OSTE) do not fully address supervisory skills.
• A gap exists in continuing medical education for effective physician supervision.

Purpose of the Study:

• To develop and validate an Objective Structured Supervision Evaluation (OSSE) for attending physicians.
• To create a tool that specifically assesses the distinct skills required for effective physician supervision.
• To address the need for standardized assessment in physician training and supervision.

Main Methods:

• An expert panel iteratively developed the OSSE instrument, a checklist for simulated endotracheal intubation scenarios.
• Trained standardized residents evaluated faculty participants using the OSSE.
• Validity was assessed using Kane's framework, incorporating simulation-based mastery learning (SBML) until a minimum passing standard (MPS) was met.

Main Results:

• The final OSSE included 19 items across domains like supervision climate, patient care control, and instructional skills.
• The assessment tool demonstrated excellent reliability (ICC 0.84-0.89) and good internal consistency (Cronbach's α = 0.813).
• 70.8% of participants met the MPS initially, with all achieving it after SBML, showing a significant average score increase of 19.5%.

Conclusions:

• The OSSE is a promising tool for evaluating faculty supervision performance.
• Further research should investigate the impact of OSSE training on clinical supervision and patient-centered outcomes.
• The OSSE has the potential to enhance resident teaching and improve overall patient care quality.