Predictive modeling in practice: improving the participant identification process for care management programs using...

The objective of this study was to optimize predictive modeling in the participant selection process for care management (CM) programs by determining the ideal cut point selection method. Comparisons included: (a) an evidence-based "optimal" cut point versus an "arbitrary" threshold, and (b) condition-specific cut points versus a uniform screening method. Participants comprised adult Medicaid health plan members enrolled during the entire study period (January 2007-December 2008) who had at least 1 of the chronic conditions targeted by the CM programs (n = 6459). Adjusted Clinical Groups Predictive Modeling (ACG-PM) system risk scores in 2007 were used to predict those with the top 5% highest health care expenditures in 2008. Comparisons of model performance (ie, c statistic, sensitivity, specificity, positive predictive value) and identified population size were used to assess differences among 3 cut point selection approaches: (a) single arbitrary cut point, (b) single optimal cut point, and (c) condition-specific optimal cut points. The "optimal" cut points (ie, single and condition-specific) both outperformed the "arbitrary" selection process, yielding higher probabilities of correct prediction and sensitivities. The condition-specific optimal cut point approach also exhibited better performance than applying a single optimal cut point uniformly across the entire population regardless of condition (ie, a higher c statistic, specificity, and positive predictive value, although sensitivity was lower), while identifying a more manageable number of members for CM program outreach. CM programs can optimize targeting algorithms by utilizing evidence-based cut points that incorporate condition-specific variations in risk. By efficiently targeting and intervening with future high-cost members, health care costs can be reduced.