Profiling provider outcome quality for pay-for-performance in the presence of missing data: a simulation approach

Area of Science:

• Health Services Research
• Health Outcomes
• Mental Health Services

Background:

• Pay-for-performance programs increasingly use provider profiling based on patient outcomes.
• Missing patient data due to loss to follow-up can compromise the accuracy of outcome profiling, especially for chronic conditions.
• Accurate provider profiling is essential for effective quality improvement in healthcare.

Purpose of the Study:

• To assess the impact of missing data on the accuracy of different provider outcome profiling approaches in primary care depression treatment.
• To compare the performance of relative (tournament-style) versus absolute (fixed threshold) profiling methods under various missing data scenarios.

Main Methods:

• Utilized data from two large initiatives (IMPACT and D déprimé) to create parameters for a Monte Carlo simulation.
• Simulated patient remission rates for major depression at 6 months.
• Evaluated two profiling approaches: relative (80th percentile) and absolute (30% remission rate).
• Partitioned total error into components from random sampling and missing data.

Main Results:

• Relative profiling approaches demonstrated approximately 20% lower total error rates compared to absolute profiling.
• Missing data contributed less to the total error in relative profiling (11-21%) versus absolute profiling (16-33%).
• Absolute profiling was significantly more sensitive to missing data correlated with depression remission.

Conclusions:

• Relative profiling methods offer superior accuracy and robustness to missing data in pay-for-performance programs.
• These findings are critical for designing reliable quality measurement systems in healthcare, particularly for conditions with potential patient attrition.