Can managed care plans reliably infer the quality of cardiac surgeons' outcomes?

To assess whether managed care plans can reliably infer the quality of cardiac surgeons' outcomes.

Evaluation of administrative discharge data and reported health plan enrollments.

We analyzed 221,327 coronary artery bypass graft (CABG) admissions performed by 398 cardiac surgeons in 75 state-regulated hospitals in Florida between 1998 and 2006. For our outcomes quality measure, we constructed surgeon-level risk-adjusted mortality rates using demographic and comorbidity data. We also obtained managed care plan enrollments in Florida in 2005 to discern the number of patient outcomes possibly seen by any individual plan. Finally, we constructed a confidence interval around any particular surgeon's CABG outcomes quality and tested whether the surgeon's quality could reliably be found to be worse than benchmarks using normal approximations and exact binomial limits.

Even if a plan had as high as a 50% share of a county's managed care-insured CABG patients, then of 86 surgeons in the 5 largest counties in Florida, no surgeon could confidently be judged to be of poorer than average quality.

In cardiac surgeons' outcomes quality monitoring, individual managed care plans face a "law of small numbers." Insufficient patient volume by contracted surgeons, inadequate variation in outcomes, and low levels of adverse outcomes combine to make true quality almost impossible to infer. Some mitigation may be possible through more effective use of data (more measures and pooling over time) and through more effective interorganizational sharing of data (leveraging specialist society quality data and statewide pooling).