Public Disclosure of Results From Artificial Intelligence/Machine Learning Research in Health Care: Comprehensive...

Despite the rapid growth of research in artificial intelligence/machine learning (AI/ML), little is known about how often study results are disclosed years after study completion.

We aimed to estimate the proportion of AI/ML research that reported results through ClinicalTrials.gov or peer-reviewed publications indexed in PubMed or Scopus.

Using data from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov, we identified studies initiated and completed between January 2010 and December 2023 that contained AI/ML-specific terms in the official title, brief summary, interventions, conditions, detailed descriptions, primary outcomes, or keywords. For 842 completed studies, we searched PubMed and Scopus for publications containing study identifiers and AI/ML-specific terms in relevant fields, such as the title, abstract, and keywords. We calculated disclosure rates within 3 years of study completion and median times to disclosure-from the "primary completion date" to the "results first posted date" on ClinicalTrials.gov or the earliest date of journal publication.

When restricted to studies completed before 2021, ensuring at least 3 years of follow-up in which to report results, 7.0% (22/316) disclosed results on ClinicalTrials.gov, 16.5% (52/316) in journal publications, and 20.6% (65/316) through either route within 3 years of completion. Higher disclosure rates were observed for trials: 11.0% (15/136) on ClinicalTrials.gov, 25.0% (34/136) in journal publications, and 30.1% (41/136) through either route. Randomized controlled trials had even higher disclosure rates: 12.2% (9/74) on ClinicalTrials.gov, 31.1% (23/74) in journal publications, and 36.5% (27/74) through either route. Nevertheless, most study findings (79.4%; 251/316) remained undisclosed 3 years after study completion. Trials using randomization (vs nonrandomized) or masking (vs open label) had higher disclosure rates and shorter times to disclosure. Most trials (85%; 305/357) had sample sizes of ≤1000, yet larger trials (n>1000) had higher publication rates (30.8%; 16/52) than smaller trials (n≤1000) (17.4%; 53/305). Hospitals (12.4%; 42/340), academia (15.1%; 39/259), and industry (13.7%; 20/146) published the most. High-income countries accounted for 82.4% (89/108) of all published studies. Of studies with disclosed results, the median times to report through ClinicalTrials.gov and in journal publications were 505 days (IQR 399-676) and 407 days (IQR 257-674), respectively. Open-label trials were common (60%; 214/357). Single-center designs were prevalent in both trials (83.3%; 290/348) and observational studies (82.3%; 377/458).

For nearly 80% of completed studies, findings remained undisclosed within the 3 years of follow-up, raising questions about the representativeness of publicly available evidence. While methodological rigor was generally associated with higher publication rates, the predominance of single-center designs and high-income countries may limit the generalizability of the results currently accessible.