Optimal statistical decisions for hospital report cards

Area of Science:

• Health Services Research
• Decision Analysis
• Biostatistics

Background:

• Hospital report cards aim to inform patient and provider decision-making regarding healthcare quality.
• Evaluating hospital performance necessitates careful consideration of statistical methods to avoid misclassification.

Purpose of the Study:

• To analyze hospital report card design within a decision-theoretic framework.
• To determine the implications of significance level choices on misclassification costs.
• To identify optimal significance levels for specific cost functions in hospital performance evaluation.

Main Methods:

• Utilized a theoretical hospital mortality model to compute false positive and false negative rates.
• Determined implicit cost functions associated with significance levels of 0.05 and 0.01.
• Calculated optimal statistical significance levels to minimize predefined cost functions for hospital misclassification.

Main Results:

• Lower significance levels reduce the relative cost of false negatives compared to false positives.
• Increased patient volume or proportion of poor-performing hospitals decreases the relative cost of false negatives.
• Significance levels of 0.05 or 0.01 are suboptimal for cost functions heavily penalizing false negatives.

Conclusions:

• Standard significance levels (0.05, 0.01) in hospital report cards imply specific, potentially suboptimal, cost functions.
• Optimal cost function values for identifying high-mortality hospitals vary depending on patient numbers and true quality distribution.