Methods for evaluating living guidelines: Early guidance based on practical experience

To provide guidance for designing and conducting process and impact evaluations for living guidelines; drawing on the practical experiences of the Australian Living Evidence Collaboration (ALEC) and its member groups.

In this paper, we describe and reflect upon the methods for evaluating living guidelines, providing practical advice drawing on our experiences conducting diverse evaluations of three living guidelines developed by the ALEC and its member groups - the Australian and New Zealand Clinical Guidelines for Stroke Management, the Australian Guidelines for the Clinical Care of people with COVID-19, and the Australian Pregnancy and Postnatal Care guidelines. We describe and outline preliminary methods for 1) process evaluations to assess and refine living guideline processes and outputs as guideline development is underway; and 2) impact evaluations to explore if and how living guidelines are being used, their value and the impacts they are having. Our guidance includes economic evaluations to assess the economic costs and benefits of developing and maintaining living guidelines.

Process and impact evaluations have been pivotal in underpinning improvements in the ALEC living guideline methods and processes, including the format, content and dissemination of the living guidelines, and our engagement with key stakeholders. Conducting thorough evaluations throughout the cycle of living guideline development and maintenance provides crucial information about the facilitators and enablers, strengths and weaknesses in the process of living guideline production and implementation. The methods described here can be tailored by guideline development programs and applied to evaluate living guidelines across diverse topics and settings.