Reliability of utilization measures for primary care physician profiling

Area of Science:

• Health Services Research
• Medical Informatics
• Healthcare Quality Improvement

Background:

• Rising healthcare costs drive interest in physician utilization measures for profiling.
• Reliability and accuracy of these measures in capturing true physician differences are not well-studied.

Purpose of the Study:

• To assess the reliability of 11 physician utilization measures used for primary care physician (PCP) feedback.
• To determine if these measures accurately reflect real variations among physicians and practices.

Main Methods:

• Utilized 2008-2010 claims data from a New York health improvement organization.
• Analyzed data from 2938 PCPs serving 853,187 patients.
• Measured reliability using hierarchical models and test-retest methods, with scores ranging from 0 (random noise) to 1 (real variation).

Main Results:

• High reliability (mean > 0.85) was found for measures of PCP visits, specialty visits, PCP lab tests, and PCP radiology/other tests at both physician and practice levels.
• Lower reliability was observed for measures including emergency room visits, hospital admissions, readmissions, and various care facility visits, indicating substantial measurement error.

Conclusions:

• Certain utilization measures are reliable for profiling primary care physicians and practices.
• Caution is advised when using less reliable measures for public reporting or pay-for-performance incentives due to potential measurement error.