Expert panel as reference standard procedure in diagnostic accuracy studies: a systematic scoping review and...

Area of Science:

• Medical research methodology
• Diagnostic accuracy studies
• Expert panel utilization

Background:

• Expert panels serve as a reference standard in diagnostic accuracy studies when a definitive test is unavailable.
• Methodological approaches and reporting quality of expert panels vary significantly across studies.
• This review examines the current landscape of expert panels used as reference standards in diagnostic research.

Purpose of the Study:

• To map the current landscape of expert panels used as reference standards in diagnostic accuracy or model studies.
• To identify variations in the design, analysis, and reporting of expert panel procedures.
• To highlight the need for standardized guidelines for expert panel methodology.

Main Methods:

• Systematic literature search of PubMed for studies published between June 1, 2012, and October 1, 2022.
• Inclusion criteria: studies assessing diagnostic accuracy using an expert panel as a reference standard, reported in English, Dutch, or German.
• Data extraction focused on study characteristics, expert panel composition, and methodology, with double-checking for accuracy.

Main Results:

• 318 studies were included, with expert panels used across various medical fields, predominantly oncology (20%).
• Panel sizes varied: 2 or fewer experts (29%), 3-4 experts (55%), 5+ experts (16%).
• Diverse panel types were employed (independent, consensus, staged, tiebreaker), with 21% of studies having unclear decision-making processes.

Conclusions:

• Significant heterogeneity exists in expert panel composition and methodology within diagnostic research.
• Inconsistent reporting of key methodological characteristics impedes study replication and may mask biases.
• There is a critical need for improved guidance and reporting standards (e.g., extensions to STARD and TRIPOD) for expert panel procedures.