Panel stacking is a threat to consensus statement validity

Area of Science:

• Public Health
• Medical Policy
• Evidence Synthesis

Background:

• Consensus statements significantly influence medical and public health decisions.
• Some statements lack systematic evidence synthesis, relying on expert panels and Delphi processes.
• Potential bias from panel member advocacy poses a threat to consensus validity.

Purpose of the Study:

• To analyze how consensus statements can be misleading.
• To examine a recent COVID-19 Delphi consensus statement as a case example.
• To propose improvements for the validity of consensus statements.

Main Methods:

• In-depth analysis of a high-impact Delphi consensus statement on COVID-19.
• Identification and assessment of panel member advocacy and potential conflicts of interest.
• Evaluation of declared conflicts versus actual advocacy positions.

Main Results:

• Many selected panel members, including 35% of the core panel, advocated for COVID-19 elimination (Zero-COVID).
• These members were leaders in aggressive advocacy groups and their conflicts were largely undeclared.
• The case example demonstrated significant bias due to undeclared advocacy.

Conclusions:

• Consensus statements require rigorous evidence synthesis to be valid.
• Maximal transparency regarding potential biases, including advocacy and lobbying, is essential.
• Biased impact from advocacy groups on consensus panels must be avoided.