Implementing and using quality measures for children's health care: perspectives on the state of the practice

The objective of this study was to identify issues, obstacles, and priorities related to implementing and using child health care quality measures from the perspectives of 4 groups: 1) funders of quality-measurement development and implementation; 2) developers of quality measures; 3) users of quality measures (including Medicaid and the State Children's Health Insurance Program, employer coalitions, and consumer groups); and 4) health plans and providers (in their role as both subjects and users of quality measures).

A series of semistructured interviews was conducted with approximately 40 opinion leaders drawn from these 4 groups. The interviews were conducted by telephone between September and December of 2001. Major topic areas covered in the interviews were similar across the groups. Topic areas included 1) strategic vision and/or objectives for funding, developing, or using quality measures for children's health care; 2) a brief summary of the specific quality measures funded, developed, or used; 3) issues and challenges facing funders and developers of measures; 4) major successes achieved; 5) obstacles to implementation and use of measures; and 6) priority needs for future funding.

Leaders from all 4 groups acknowledge the importance of developing a robust set of quality measures that can serve multiple objectives and multiple audiences. Standardization of measures is viewed as a critical feature related to all objectives. An assessment of specific quality measures funded, developed, or used by strategic objective shows a high correlation between the uses intended by funders and developers and the actual applications of the various users. The most commonly cited measures across all groups are the Consumer Assessment of Health Plans Survey and Health Plan Employer Data and Information Set, followed by the Child and Adolescent Health Measurement Initiative and special topic studies to support quality-improvement applications (eg, asthma, diabetes, etc). The major issues and challenges cited in common among funders and developers are 1) the lack of trained capacity in the field to conduct needed research and development, and 2) the difficulty in generating sustained interest and support among funders because of the complexity of quality-measurement issues, competing funding priorities in the face of limited funds available to allocate, and the lack of clear and compelling evidence that quality measurement and improvement actually result in better outcomes for children. The 3 most common successes cited across all 4 groups are 1) the growing consensus and collaboration among diverse stakeholder groups involved in measurement development and implementation; 2) the increasing collection and use of specific measures; and 3) early documentation of tangible results in terms of improved quality of care. Specific measurement tools cited as successes by funders and developers include the Medicaid Health Plan Employer Data and Information Set, Consumer Assessment of Health Plans Survey, the Child and Adolescent Health Measurement Initiative, and Rand QA Tools. The most important obstacle reported across all groups is the lack of a strong and compelling "business case" that clearly demonstrates the benefits of quality measurement relative to the costs of implementation. Strongly related to this barrier is the cost of implementing and using measures without a sustainable source of financing as well as the absence of strong public awareness and political support for children's health care quality measurement. Another major barrier cited is the lack of coordination among funders, which prevents the field from developing a unified approach to addressing the numerous technical, political, and administrative issues also cited at length by the leaders interviewed. The 5 top needs for future funding identified across all 4 groups follow directly from the major obstacles that they reported: 1) develop the business case for children's health care quality measurement and improvement based on rigorous cost-bigorous cost-benefit analysis and documentation of quantifiable successes; 2) develop new measures to fill the gaps in critical areas (including adolescent health care, behavioral health, and chronic conditions) that can be applied at the hospital and ambulatory care provider levels; 3) invest in building needed research capacity, a trained pool of users of quality measures, and the capacity among providers to understand and use quality-improvement methods and tools; 4) invest in developing an information infrastructure that will support the efficient collection and use of measures for multiple purposes, including clinical practice, quality measurement, and quality improvement; and 5) develop increased public awareness and support for quality measurement based on improved strategies for communicating with consumers, purchasers, providers, and policy makers.

Several implications are suggested by these perspectives for the future direction of quality measurement in children's health care. First, to meet the funding needs identified, many funders must improve coordination to reduce the noise and fragmentation generated by numerous competing or redundant activities. Improved coordination among funders will help assure maximum impact and the efficient use of scarce resources. Second, the importance attached to standardization of measures by both users and developers may conflict at times with the need for innovation and flexibility. Child health quality leaders will need to manage this tension between standardization and innovation to maintain an appropriate balance between the benefits of both. Finally, many of the obstacles identified are not unique to children's health care. Child health quality leaders will need to determine to what extent their efforts to overcome these obstacles can be successfully undertaken independently as opposed to in concert with groups concerned about other populations and sectors in the health care system.