"Our complication rates are lower than theirs": statistical critique of heart valve comparisons

From the widely differing complication rates published for every heart valve, is it possible to determine a true rate for each valve and to compare the rates of two different valves? We investigated this question for the two most popular bileaflet valves.

Aortic valve data were abstracted from 14 St Jude Medical (St Jude Medical Inc, Minneapolis, Minn; 33,125 patient-y) and 11 Carbomedics (Sulzer Carbomedics Inc, Austin, Tex; 19,141 patient-y) series, and mitral valve data were abstracted from 11 St Jude Medical (21,553 patient-y) and 8 Carbomedics (8368 patient-y) series. Regression analysis was used to accommodate heterogeneity among rates with the same valve model, to estimate hazard ratios and 95% confidence intervals for the valve model effect, and to incorporate other series-level risk factors.

Most of the complication rates with both valve models exhibited significant heterogeneity. For thromboembolism and bleeding, the relative risks for valve model were not significantly different from unity. Valve thrombosis rates exhibited less heterogeneity: the Carbomedics valve had a lower rate in the aortic position (hazard ratio 0.2, 95% confidence interval 0.1-0.6) and a higher rate in the mitral position (hazard ratio 1.9, 95% confidence interval 1.0-3.8).

Unlike simple weighted averages of valve complication rates, regression methods can incorporate heterogeneity related to center effects and allow for inclusion of other risk factors. Thromboembolism and bleeding rates were not significantly different with St Jude Medical and Carbomedics valves. Valve thrombosis appeared to differ between the two valves, but the absolute differences in rates were small. Because of the variability among rates with the same valve, statistical comparisons must be interpreted cautiously.