Retooling of Paper-based Outcome Measures to Electronic Format: Comparison of the NY State Public Risk Model and...

Risk adjustment is critical in the comparison of quality of care and health care outcomes for providers. Electronic health records (EHRs) have the potential to eliminate the need for costly and time-consuming manual data abstraction of patient outcomes and risk factors necessary for risk adjustment.

Leading EHR vendors and hospital focus groups were asked to review risk factors in the New York State (NYS) coronary artery bypass graft (CABG) surgery statistical models for mortality and readmission and assess feasibility of EHR data capture. Risk models based only on registry data elements that can be captured by EHRs (one for easily obtained data and one for data obtained with more difficulty) were developed and compared with the NYS models for different years.

Only 6 data elements could be extracted from the EHR, and outlier hospitals differed substantially for readmission but not for mortality. At the patient level, measures of fit and predictive ability indicated that the EHR models are inferior to the NYS CABG surgery risk model [eg, c-statistics of 0.76 vs. 0.71 (P<0.001) and 0.76 vs. 0.74 (P=0.009) for mortality in 2010], although the correlation of the predicted probabilities between the NYS and EHR models was high, ranging from 0.96 to 0.98.

A simplified risk model using EHR data elements could not capture most of the risk factors in the NYS CABG surgery risk models, many outlier hospitals were different for readmissions, and patient-level measures of fit were inferior.