When to base clinical policies on observational versus randomized trial data

Area of Science:

• Health Services Research
• Clinical Policy Development
• Evidence-Based Medicine

Background:

• Physicians face challenges in determining sufficient evidence for adopting new clinical policies.
• Analysis of large clinical and administrative databases is increasingly used for policy changes.
• Database analysis has limitations in controlling confounding variables, risking incorrect causal inferences.

Purpose of the Study:

• To review the advantages and disadvantages of using large database analysis versus randomized studies for clinical policy decisions.
• To present a method for assessing the costs and benefits of conducting randomized studies.
• To inform healthcare organizations on evidence-based policy adoption.

Main Methods:

• Comparative review of evidence-based methodologies for clinical policy.
• Description and illustration of a technique for cost-benefit analysis of randomized studies.
• Case study of a physician-managed health care organization's decision-making process.

Main Results:

• Randomized studies offer superior protection against adopting inferior policies compared to database analysis.
• Randomized studies present challenges including increased costs and uncertain direct patient benefit.
• A cost-benefit analysis technique can aid in justifying the initiation of randomized studies.

Conclusions:

• The decision to adopt new clinical policies requires balancing evidence strength with practical considerations.
• Randomized studies, despite their challenges, are crucial for establishing causality and ensuring policy effectiveness.
• A structured cost-benefit analysis supports informed decisions on sponsoring randomized studies for quality improvement.