Case finding with incomplete administrative data: observations on playing with less than a full deck

Capacity constraints and efficiency considerations require that disease management programs identify patients most likely to benefit from intervention. Predictive modeling with available administrative data has been used as a strategy to match patients with appropriate interventions. Administrative data, however, can be plagued by problems of incompleteness and delays in processing. In this article, we examine the effects of these problems on the effectiveness of using administrative data to identify suitable candidates for disease management, and we evaluate various proposed solutions. We build prospective models using regression analysis and evaluate the resulting stratification algorithms using R² statistics, areas under receiver operator characteristic curves, and cost concentration ratios. We find delays in receipt of data reduce the effectiveness of the stratification algorithm, but the degree of compromise depends on what proportion of the population is targeted for intervention. Surprisingly, we find that supplementing partial data with a longer panel of more outdated data produces algorithms that are inferior to algorithms based on a shorter window of more recent data. Demographic data add little to algorithms that include prior claims data, and are an inadequate substitute when claims data are unavailable. Supplementing demographic data with additional information on self-reported health status improves the stratification performance only slightly and only when disease management is targeted to the highest risk patients. We conclude that the extra costs associated with surveying patients for health status information or retrieving older claims data cannot be justified given the lack of evidence that either improves the effectiveness of the stratification algorithm.