A Multifaceted Organizational Physician Assessment Program: Validity Evidence and Implications for the Use of...

Area of Science:

• Medical Education
• Healthcare Quality Improvement
• Physician Performance Measurement

Background:

• Assessing physician performance is crucial for quality healthcare.
• Existing assessment programs often lack comprehensive validity evidence.
• Evaluating different scoring methods is essential for accurate performance evaluation.

Purpose of the Study:

• To provide validity evidence for a multifaceted physician performance assessment program.
• To evaluate the impact of mean versus top box scoring on performance data.
• To offer practical guidance for implementing physician assessment strategies.

Main Methods:

• Physicians from internal medicine, neurology, and psychiatry practices were included.
• Data collected included patient complaints/compliments, note-signing timeliness, cost of care, and survey results.
• Survey data were analyzed using both mean scores and top box percentages.

Main Results:

• Physician performance varied across metrics, with high on-time note signing and above-average costs.
• Survey scores showed moderate reliability and associations between learner and patient feedback.
• Top box scores demonstrated greater variability among physicians compared to mean scores.

Conclusions:

• The study provides validity evidence for seven common physician performance assessments.
• Top box scoring effectively amplifies performance distinctions among high-achieving physicians.
• Findings support informed use of physician performance data and assessment program implementation.