Benchmarking physician performance: reliability of individual and composite measures

Area of Science:

• Health Services Research
• Quality Improvement
• Health Informatics

Background:

• Physician performance measures are crucial for quality improvement, contracting, and incentives.
• Concerns exist regarding the methodological challenges and reliability of these measures.
• Administrative data is increasingly used for physician profiling.

Purpose of the Study:

• To assess the reliability of quality measures used for physician performance evaluation.
• To determine if administrative data provides adequate numbers of quality events for reliable measurement.
• To inform the use of quality data in public reporting and accountability.

Main Methods:

• Evaluation of administrative claims and enrollment data from 9 health plans (over 11 million patients).
• Calculation of quality events, mean performance, and reliability estimates for 27 quality measures.
• Estimation of reliability using variance components; 0.70 considered adequate.

Main Results:

• Only 10 quality measures met the reliability threshold (0.70) with at least 50 quality events.
• Reliability remained low for many measures even with 50 quality events.
• Composite scores improved reliability but still evaluated a limited proportion of physicians.

Conclusions:

• Most physicians lack sufficient quality events in administrative data for reliable individual quality measure assessment.
• Reliability of quality measures is a critical consideration for public reporting and accountability.
• Improvements in data availability for physician profiling are necessary.