Sample size in multistakeholder Delphi surveys: at what minimum sample size do replicability of results stabilize?

Area of Science:

• Health Research Methodology
• Survey Science
• Clinical Trials

Background:

• The optimal sample size for multistakeholder Delphi surveys is not well-established.
• Delphi surveys are increasingly used in health research for consensus building.

Purpose of the Study:

• To determine the minimum sample size for achieving reliable results in multistakeholder Delphi surveys.
• To assess the impact of sample size and participant characteristics on result replicability.

Main Methods:

• Utilized data from three large international Delphi surveys (SPIRIT, CONSORT-SPI, burn care core outcome set).
• Employed resampling with replacement to generate subsamples and compare median values.
• Calculated percentage replicability and variability based on median agreement with full datasets.

Main Results:

• An average replicability of 81% (10% variability) was achieved with a sample size of 60 participants.
• A sample size of 80 participants reached ≥80% replicability in one dataset.
• Sample sizes of 20-30 participants yielded moderate replicability (64-77%) for subgroup analyses.

Conclusions:

• A minimum sample size of 60-80 participants is recommended for high replicability (≥80%) in multistakeholder Delphi surveys.
• For Delphi studies focusing on individual stakeholder groups, 20-30 participants per group may be adequate.