A methodological review finds that the statistical analysis of most comparative diagnostic accuracy studies had...

Diagnostic accuracy studies that compare two or more index tests (comparative diagnostic accuracy studies) can be at risk of generating biased results due to shortcomings in the statistical analysis. Understanding which statistical methods are used and how they are described in such studies is therefore crucial.

To evaluate statistical methods used in comparative diagnostic accuracy studies.

Methodological review.

We searched PubMed for comparative accuracy systematic reviews published in 2023. Of the primary studies included in these reviews, a subset of 200 comparative diagnostic accuracy studies was randomly selected. Seven reviewers extracted data in pairs about study design features, sample size calculation, statistical analysis methods used for comparing diagnostic accuracy, and methods for dealing with missing data and confounding.

Of 200 comparative diagnostic accuracy studies, 184 (92%) had a fully paired design, 6 (3%) had a partially paired design, and 10 (5%) had an unpaired design. Eighty-four (42%, 95% CI: 35.1%-49.2%) studies did not statistically compare diagnostic accuracy estimates (i.e. no confidence interval or statistical test for the difference was reported), while this was unclear in two (1%) studies. Of the remaining 114 studies including a statistical comparison, 44% (50/114, 95% CI: 34.6%-53.5%) failed to report the relevant statistical methods and of those that did report their methods, 17.2% (11/64, 95% CI: 8.9%-28.7%) used at least one inappropriate statistical method. Sixty-nine of 200 (35%, 95% CI: 27.9%-41.5%) studies reported the presence of missing, indeterminate, or intermediate test results and nearly all (96%, 66/69, 95% CI: 87.8%-99.1%) conducted complete case analysis. While 14 (7%, 95% CI: 3.9%-11.5%) studies used neither a fully paired nor randomized design, none of these reported adjusting for confounders. Overall, most studies (71%, 95% CI: 64.2%-77.2%) showed one or more shortcomings in the statistical analysis.

The majority of comparative diagnostic accuracy studies suffer from suboptimal analyses, which may increase the risk of misleading or overconfident conclusions, while the incomplete reporting of statistical methods hampers interpretation. Researchers should carefully consider the choice of statistical methods for comparative accuracy and adhere to the STARD 2015 (Standards for Reporting of Diagnostic Accuracy Studies) reporting guidelines.

We assessed how statistical methods are used and reported in studies comparing diagnostic tests (comparative diagnostic accuracy studies). We examined 200 such studies published between 1995 and 2023. Overall, most studies (142/200, 71%) had at least one shortcoming in the statistical analysis. This finding suggests that statistical analyses in comparative diagnostic accuracy studies are often suboptimal.