Determining resident clinical performance: getting beyond the noise.

Area of Science:

• Medical Education
• Anesthesiology
• Clinical Assessment

Background:

• Accurate assessment of resident clinical skills is crucial for medical training, advancement, and improvement.
• Distinguishing between physician competency (what they can do) and performance (what they do) is key.
• There is a continuous demand for dependable methods to evaluate resident clinical performance.

Purpose of the Study:

• To develop and validate a robust system for assessing anesthesia resident clinical performance.
• To address the need for reliable and valid measures in graduate medical education.

Main Methods:

• Anesthesia residents underwent weekly confidential evaluations by supervising faculty.
• Evaluations included ratings on core competencies, rater confidence, and qualitative comments.
• A novel scoring normalization technique using Z-scores was implemented to correct for faculty-specific scoring biases.

Main Results:

• Over 14,000 evaluations were collected from 108 residents over two academic years.
• Raw faculty scores exhibited significant positive bias and inconsistent scoring ranges.
• Normalization using Z-scores effectively eliminated bias, yielding stable and reliable performance metrics.
• Normalized scores correlated with external knowledge assessments and identified residents needing intervention.

Conclusions:

• A reliable and valid system for assessing resident clinical performance was successfully demonstrated.
• The developed system supports objective evaluation across all training levels.
• This method enhances the accuracy of resident performance assessment in anesthesiology.