Using the State Plan Index to evaluate the quality of state plans to prevent obesity and other chronic diseases

Implicit in public health planning models is the assumption that good public health plans lead to good programs, and good programs lead to desired health outcomes. Despite considerable resources that are devoted to developing plans, public health agencies and organizations have lacked a tool for evaluating the finished product of their planning efforts -- the written plan itself -- as an important indicator of progress. To address the need for an instrument to assess the quality of state plans designed to prevent and control chronic diseases, we created and tested the State Plan Index and used it to evaluate the quality of nine state plans aimed at preventing and reducing obesity.

The State Plan Index was developed under the auspices of the Centers for Disease Control and Prevention (CDC) in collaboration with public health experts in federal, state, and academic settings. The State Plan Index included 55 items related to plan quality arranged into nine components. Each item was rated on a Likert scale from 0 to 5, with 5 being the highest rating. Each plan also received a separate overall plan quality score using the same scale. Each state plan was evaluated by four or five raters using the State Plan Index. For each plan, the 55 items were averaged to calculate an item average score, and a subscore was calculated for each State Plan Index component. Finally, five states also self-rated their own plans (self score).

The mean item average score for all plans was 2.4 out of 5.0. The range of item average scores was 1.0 to 3.0. The component of the State Plan Index with the highest mean component score (3.3) was Presentation of Epidemiologic Data on Disease Burden. The components with the lowest component scores were Resources for Plan Implementation (0.7); Integration of Obesity Efforts with Other Chronic Disease Efforts (1.7); and Program Evaluation (2.0). Plan quality was rated higher when based on the single overall plan quality score assigned by raters. In addition, self scores were consistently and substantially higher than rater-assigned scores.

Evaluation of plans early in the life of programs can be used to strengthen existing programs and to guide programs newly engaged in chronic disease prevention planning. The CDC has used the State Plan Index evaluation results to guide technical assistance, plan training sessions, and enhance communication with state staff about plan content, quality, and public health approach. Some state program directors self-evaluated their obesity draft plan and used the evaluation results to strengthen their planning process and to guide plan revisions. Other states have adapted the State Plan Index as a framework for new planning efforts to prevent obesity as well as other chronic diseases.