Mitigating imperfect data validity in administrative data PSIs: a method for estimating true adverse event rates.

Area of Science:

• Health Services Research
• Medical Informatics
• Quality Improvement

Background:

• Patient Safety Indicator (PSI) report cards are crucial for healthcare quality assessment.
• Imperfect data validity in PSI reports can lead to inaccurate comparisons and misinformed decisions.
• Existing methods often fail to account for variations in data quality across hospitals.

Purpose of the Study:

• To develop and evaluate a methodological framework for mitigating data validity challenges in PSI report cards.
• To estimate the true hospital-level adverse event rates by adjusting for data quality issues.
• To provide a more accurate basis for health system report cards.

Main Methods:

• A framework combining existing PSI algorithms, medical record review for data validity assessment, and Bayesian calculations was applied to simulated data.
• Three hospitals with varying data quality were simulated to test the framework.
• A small random sample of charts was used to measure hospital-specific data validity.

Main Results:

• The framework successfully estimated true PSI rates, revealing significant discrepancies between measured and true rates.
• For hospitals with moderate data quality, estimated true rates could be triple the measured rates (e.g., 1.4% vs. 0.5%).
• Hospitals with poor data quality showed substantially higher estimated true rates, up to 50-fold differences (e.g., 5.0% vs. 0.1%).

Conclusions:

• Combining medical chart review with Bayesian analysis offers a robust approach to estimating true adverse event rates.
• This method enhances the accuracy of health system report cards by addressing data validity limitations.
• Accurate reporting of patient safety metrics is essential for effective quality improvement initiatives.