Assessing return on investment of defined-population disease management interventions

Strategies to reduce health expenditures through the improvement of health and quality of care are in high demand. A group of experts formed a nonpartisan, independent work group, under the sponsorship of the National Managed Health Care Congress. Its goal was to establish a list of easy-to-understand, actionable, and usable recommendations to enable disease management program advocates to conduct basic-level evaluations.

The work group made recommendations concerning identification of reference and intervention population, population definitions, quantitative methods and data quality, confounding and bias, and stakeholder agreements/contracting.

A case study was created to quantitatively illustrate some of the major issues raised by the work group. Five typical errors were simulated by applying different rules to the intervention population than to the reference population: differential inclusion (high versus low risk), differential exclusion (high versus low risk) and differential claims run-out. Compared with the true impact, four of the five errors resulted in a bias toward "intervention effect," while one (differential inclusion of high-risk patients) was biased against the "intervention effect." The direction and magnitude of the bias in natural settings will not necessarily follow this pattern.