Trustworthiness and transparency features were less frequent in randomized trials presenting large effects for...

Large effect sizes (ESs), especially when prominently presented in trial abstracts, draw large attention, but it is important to understand whether they are trustworthy. We aimed to assess indicators of transparency and trustworthiness in randomized controlled trials (RCTs) reporting some large ES for continuous outcomes in their abstract, in comparison with RCTs presenting only nonlarge ESs in their abstract.

We included RCTs indexed in MEDLINE between January 1, 2024 and March 18, 2025, presenting at least one standardized mean differences of absolute value 0.8 or higher (large ES) vs those presenting only smaller absolute standardized mean differences in their abstract. Trial characteristics and methodological features were extracted systematically in large ES and nonlarge ES trials. Primary outcome was prespecified protocol registration, secondary outcomes were any protocol registration and public availability or repository placement of raw data (preregistered protocol: https://osf.io/8xasw).

We evaluated 152 trials with large ESs in their abstract and 175 trials with only nonlarge ESs in their abstract. Large ES trials had suggestively lower rates of preregistered protocols (45% vs 61%, P = .0054) and significantly lower rates of any protocol registration (74% vs 87%, P = .0028) than nonlarge ES trials. There was no difference in raw data public availability or repository placement (6% vs 7%). Large ES trials were also less likely to be multicenter (P = .0042), to have high-income country of corresponding author (P = .0001), to be conducted in high-income country site(s) (P = .0003), to have a published statistical analysis plan (P = .0216), to result from between-group comparisons (P < .0001), and tended to report less frequently allocation concealment (P = .0351). Large effects were significantly more likely to involve nondrug/nonpsychological interventions (P = .0001).

RCTs presenting large ESs in their abstracts are more likely to lack transparency and trustworthiness features and may operate with higher risk of lack of credibility.

When a medical study reports that a treatment has a big effect that naturally grabs attention. However, are impressive-sounding results actually reliable? This study investigated whether clinical trials claiming large treatment effects in their abstracts are as trustworthy as those reporting only more modest results. We compared two groups of recently published randomized controlled trials: 152 trials that reported large effects in their abstracts, and 175 trials that reported only smaller effects. We looked at several markers of good scientific practice, particularly whether researchers had registered their study plans in advance and whether they made their data openly available. We found that trials reporting large effects were less likely to have preregistered their methods and more likely to have no registered protocol at all. The two groups also differed in other aspects of trustworthiness. Large-effect trials were less likely to involve multiple research centers, less likely to have researchers from high-income countries, and less likely to have a published statistical analysis plan. They also tended to test different types of treatments-large effects appeared more often in studies of interventions other than drugs or psychological therapies. Both groups were equally poor at making their raw data publicly available (only about 6%-7%). Overall, when a trial claims to have found impressive treatment effects, some extra scrutiny may be warranted. Before trusting eye-catching results, readers should assess whether trials followed proper scientific checks and transparency safeguards.