Most methodological characteristics do not exaggerate effect estimates in nutrition RCTs: findings from a...

To evaluate the influence of bias from methodological characteristics on intervention effect estimates in nutrition randomized controlled trials (RCTs) using the Cochrane Risk Of Bias 2 (RoB2) tool.

RCTs published until 2020 were selected from a representative sample of 183 nutrition meta-analyses. Pairs of reviewers conducted data extraction and risk of bias (RoB) assessments. Average estimates due to bias (ratio of risk ratio [RRR]) were computed through meta-analyses using a random-effects model, comparing RCTs rated as "high risk" or "some concerns" to those rated as "low risk" of bias. Subgroup analyses explored differences across RoB domains, types of interventions, and outcomes. Heterogeneity was assessed through I² and τˆ², and prediction intervals were calculated.

We included 26 meta-analyses, encompassing 82 RCTs with 120 outcome-specific RoB assessments. Of these, 70% were rated as "some concerns", 18.3% as "low risk", and 11.7% as "high risk" of bias. Overall RoB did not affect intervention effect estimates (RRR 0.99, 95% CI 0.85-1.14; I² = 36%; heterogeneity estimator [τˆ²] = 0.03; prediction interval [PI] 0.66-1.47). Most RoB domains did not reveal differences in effect estimates, except for trials with biases related to deviations from the intended intervention (RRR 1.29, 95% CI 1.13-1.48; I² = 2%; τˆ² = 0.01; PI 0.97-1.72). We confirmed these findings in subgroup and meta-regression analyses.

Most methodological characteristics in nutrition RCTs, as assessed by RoB2, did not overestimate or underestimate intervention effect estimates. However, the unexpected finding that biases arising from deviations from intended interventions may lead to an underestimation of effects, rather than an overestimation, requires further research.

RCTs are considered the most reliable method for determining whether an intervention is effective. However, weaknesses in study design or conduct can distort the results, a problem known as bias. The RoB2 tool helps researchers check in a structured way whether bias is present and how much it might affect the results. In this study, we looked at 82 RCTs on dietary interventions published up to 2020. We assessed 120 outcome-specific RoB assessments and identified whether they had a low, some concerns, or high RoB. We then compared the results of trials with a higher RoB to those with a low RoB to see whether bias influenced the reported treatment effects. The reviewed trials showed variable levels of RoB and had little overall impact on the trial results. The main exception was how well participants followed the assigned diet and whether the researchers used the best available analysis methods. Problems in this area may have made the intervention's true effect seem smaller than it really was. Our findings suggest that studies on nutrition interventions are mostly free from major bias and their results can be considered reliable. However, how well participants followed the assigned intervention and how good the analysis methods were seemed to play an important role. More research is needed to understand how study quality influences the results of nutrition trials.