A comparison of a Bayesian vs. a frequentist method for profiling hospital performance

Area of Science:

• Health Services Research
• Biostatistics
• Medical Quality

Background:

• Accurate classification of hospital performance is crucial for quality improvement.
• Frequentist statistical methods are commonly used to identify outlier hospitals based on patient outcomes.
• The reliability of these frequentist methods for outlier detection is under scrutiny.

Purpose of the Study:

• To compare hospital outlier classification using frequentist versus Bayesian hierarchical models.
• To assess agreement between these statistical approaches for acute myocardial infarction 30-day mortality rates.
• To evaluate the impact of patient risk profiles and statistical thresholds on classification outcomes.

Main Methods:

• Employed a frequentist approach using logistic regression to predict mortality and identify outliers based on confidence intervals.
• Utilized Bayesian hierarchical models considering patient risk, deviation from average mortality, and probability of difference.
• Compared classification agreement using kappa statistics across various patient profiles and thresholds.

Main Results:

• Found only marginal agreement between frequentist and Bayesian methods in classifying hospital outcome outliers (kappa < 0.40 in 22 of 27 comparisons).
• Bayesian classification outcomes were sensitive to patient risk profiles, threshold differences, and probability parameters.
• The study highlights significant discrepancies in outlier identification between the two statistical frameworks.

Conclusions:

• Current frequentist methods for classifying hospital performance may lack robustness.
• Bayesian hierarchical models offer a more nuanced but complex approach to outlier detection.
• Further research is needed to determine the most meaningful methods for assessing hospital performance for stakeholders.