Validating ICD-10 Algorithms for Identifying Patient Safety Indicators Through 10,655 Charts Review

Area of Science:

• Health Services Research
• Medical Informatics
• Quality Improvement

Background:

• Patient Safety Indicators (PSIs) from administrative data are crucial for hospital quality monitoring.
• The diagnostic accuracy of ICD-10-based PSI algorithms requires validation, especially regarding sensitivity and specificity.

Purpose of the Study:

• To assess the diagnostic performance of ICD-10-CA algorithms for identifying 15 PSIs.
• To compare algorithm performance against detailed chart reviews as the reference standard.

Main Methods:

• A multicenter retrospective cohort validation study involving 10,665 adult patients across 4 acute care hospitals.
• Fifteen PSIs were identified using ICD-10-CA codes and validated via chart review.
• Diagnostic metrics included sensitivity, specificity, PPV, NPV, and overall accuracy, stratified by code type and patient characteristics.

Main Results:

• ICD-10-CA coding for any PSI demonstrated 67.0% sensitivity, 72.8% specificity, 31.6% PPV, and 92.2% NPV.
• Limiting to type II codes (post-admission conditions) increased specificity to 95.7% and PPV to 56.5% but decreased sensitivity to 29.6%.
• Algorithm validity varied by PSI and patient factors, with better performance in older adults, males, and those with higher comorbidity or adverse outcomes.

Conclusions:

• ICD-10 coded administrative data offer high specificity and NPV for PSIs but exhibit variable sensitivity and PPV.
• Using type II codes enhances PPV but compromises sensitivity, highlighting the need for adaptable coding strategies.
• Tailoring coding approaches to specific quality improvement or surveillance objectives is essential for accurate patient safety monitoring.