Setting standards for effectiveness: a comparison of expert panels and decision analysis

Area of Science:

• Cardiology
• Health Services Research
• Medical Decision Making

Background:

• Coronary revascularization procedures, including coronary artery bypass graft surgery (CABG) and percutaneous transluminal coronary angioplasty (PTCA), are common treatments for chronic stable angina.
• Establishing appropriate criteria for these interventions is crucial for optimal patient outcomes and resource allocation.

Purpose of the Study:

• To compare the appropriateness criteria for coronary revascularization developed by expert physician panels with those derived from decision analysis.
• To assess the agreement between U.S. and Dutch expert panels and between each panel and a decision analytic model.

Main Methods:

• A review of medical records for 3080 patients with chronic stable angina who underwent CABG or PTCA.
• Two expert physician panels (U.S. and Dutch) rated the appropriateness of revascularization procedures.
• Agreement was assessed between panels and with a decision analytic model's effectiveness categories using kappa statistics.

Main Results:

• Poor agreement was found between U.S. and Dutch panels for PTCA (kappa = 0.03) and slight agreement for CABG (kappa = 0.18).
• Dutch panel ratings showed substantial agreement with the decision analytic model for both PTCA (kappa = 0.83) and CABG (kappa = 0.79).
• U.S. panel ratings showed no systematic agreement for PTCA (kappa = 0.00) and poor agreement for CABG (kappa = 0.18) with the decision analytic model.

Conclusions:

• Agreement between expert panels and decision analysis varies by procedure and evidence strength.
• Dutch physicians demonstrated significantly higher agreement with decision analysis compared to U.S. physicians regarding coronary revascularization appropriateness.
• These findings suggest potential differences in the interpretation or application of evidence-based guidelines between international expert groups.