The sensitivity and specificity of forecasting high-cost users of medical care

This study compares the ability of 3 risk-assessment models to distinguish high and low expense-risk status within a managed care population. Models are the Global Risk-Assessment Model (GRAM) developed at the Kaiser Permanente Center for Health Research; a logistic version of GRAM; and a prior-expense model. GRAM was originally developed for use in adjusting Medicare payments to health plans.

Our sample of 98,985 cases was drawn from random samples of memberships of 3 staff/group health plans. Risk factor data were from 1992 and expenses were measured for 1993. Models produced distributions of individual-level annual expense forecasts (or predicted probabilities of high expense-risk status for logistic) for comparison to actual values. Prespecified "high-cost" thresholds were set within each distribution to analyze the models' ability to distinguish high and low expense-risk status. Forecast stability was analyzed through bootstrapping.

GRAM discriminates better overall than its comparators (although the models are similar for policy-relevant thresholds). All models forecast the highest-cost cases relatively well. GRAM forecasts high expense-risk status better than its comparators within chronic and serious disease categories that are amenable to early intervention but also generates relatively more false positives within these categories.

This study demonstrates the potential of risk-assessment models to inform care management decisions by efficiently screening managed care populations for high expense-risk. Such models can act as preliminary screens for plans that can refine model forecasts with detailed surveys. Future research should involve multiple-year data sets to explore the temporal stability of forecasts.