Enhancing trustworthiness checks of randomized controlled trials: a pathway to more reliable evidence

Systematic reviews (SRs) of randomized controlled trials (RCTs) are crucial for shaping clinical practice, but are only as good as the RCTs on which they are based. We aimed to summarize tools to prevent untrustworthy RCTs from entering meta-analysis. We propose six checkpoints: prepublication checks, postpublication checks, checks during aggregate-data meta-analysis, checks of individual participant data (IPD), and checks for umbrella reviews and guidelines.

We searched PubMed for tools assessing RCTs for trustworthiness.

Before publication, trialists must guarantee trustworthiness by providing registration details, protocol, datasets, and code. Journals should enhance data-sharing policies for RCTs. Research users should evaluate trials postpublication and report concerns to journals or platforms like PubPeer. Journals should be automatically alerted to comments, mark retracted references, expedite investigations, and communicate findings transparently. We found seven tools for both prepublication and postpublication checks and an international consensus set of statements for postpublication checks. Four were piloted, while one has been published as preprint (INveStigating ProblEmatic Clinical Trials in SR [INSPECT-SR]). Three were achieved by non-Delphi consensus, while INSPECT-SR used a Delphi process. No tools were formally validated; one (IPD-Integrity) underwent preliminary validation. All address trial ethics, registration, follow-up plausibility, and baseline data; 6 of 7 tools assess the plausibility of the authors' group size relative to trial context, recruitment timelines, and outcome data. Three were recommended for research user and editor checks. Systematic reviewers should assess RCTs for trustworthiness. All seven tools were recommended for systematic reviewers. Positive systematic reviewer screening for major issues can eliminate trials after study selection but before data extraction. Studies with unresolved issues should be excluded; their count should appear in the flow diagram with reasons. All tools assess study-level data. IPD-level checks should detect unexpected values and data patterns; IPD-Integrity and a developing INSPECT-SR extension can do so. Umbrella reviews should check whether original reviews assessed trustworthiness and, if needed, perform primary RCT assessments or only include SRs that have conducted checks. Guidelines should evaluate the trustworthiness of included studies.

Six levels of trustworthiness checks can prevent untrustworthy RCTs from influencing practice. At the SR level, trustworthiness assessments should be integrated in Preferred Reporting Items for Systematic Reviews and Meta-analyses.