Related Experiment Videos
Severity of Illness Index and the Adverse Patient Occurrence Index. A reliability study and policy implications
D N Schumacher1, B Parker, V Kofie
1Rockburn Institute, Elkridge, MD 21227.
Abstract:
Seven hundred fifty-two randomly selected charts from seven teaching hospitals were rated by pairs of medical record analysts. The Severity of Illness Index was unreliable with an interrater-agreement rate of 73% (kappa statistic = 0.41), and demonstrated a significant (P less than 0.0001) association with the Adverse Patient Occurrence (APO) Index. This suggests that the Severity of Illness Index is not differentiating severity of illness from quality of care. The fair to poor field reliability stems from underlying instrument subjectivity, lack of clear referent groups, and time pressure. The APO Index was also found to be unreliable (r = 0.33 and range = -0.05-0.58). Greater attention should be directed to improving objective discharge abstract, billing, and laboratory data for measuring patient severity and adverse patient occurrences.
Insights
The Severity of Illness Index showed poor reliability in measuring patient severity, conflating it with quality of care. Improvements in objective data are needed for accurate patient assessment.
Area of Science:
- Health Services Research
- Medical Informatics
- Quality Improvement
Background:
- Accurate measurement of patient severity and adverse events is crucial for healthcare quality assessment.
- Existing indices may suffer from subjectivity and lack of clear criteria, impacting reliability.
Purpose of the Study:
- To evaluate the reliability and validity of the Severity of Illness Index.
- To assess the Adverse Patient Occurrence (APO) Index.
- To identify areas for improvement in measuring patient severity and adverse events.
Main Methods:
- Randomly selected patient charts (n=752) from seven teaching hospitals were reviewed.
- Medical record analysts rated charts using the Severity of Illness Index and APO Index.
- Interrater reliability was assessed using kappa statistics and correlation coefficients.
Main Results:
- The Severity of Illness Index demonstrated poor interrater reliability (kappa = 0.41), indicating inconsistency.
- A significant association was found between the Severity of Illness Index and the APO Index (P < 0.0001), suggesting conflation of severity and quality.
- The APO Index also showed low reliability (r = 0.33).
Conclusions:
- The Severity of Illness Index is unreliable for differentiating patient severity from quality of care.
- Instrument subjectivity, lack of clear referent groups, and time pressure contribute to poor reliability.
- Enhancing objective data sources like discharge abstracts, billing, and laboratory data is recommended for improved measurement.