Risk of bias and magnitude of effect in orthodontic randomized controlled trials: a meta-epidemiological review

To assess the risk of bias (RoB) in a subset of randomized controlled trials (RCTs) published in orthodontic journals using the Cochrane RoB tool and to identify associations between domain RoB assessment and treatment effect estimates.

Fifty consecutive issues of four major orthodontic journals were electronically searched to identify RCTs. The quality of the included studies was assessed using the Cochrane RoB tool, which involves seven domains rated as 'low', 'unclear' or 'high': random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting, and other threats to internal validity. Estimates and confidence intervals (CIs) were recorded or calculated where possible for binary and continuous outcome measures. Meta-regression models were employed to assess the impact of RoB per domain on the magnitude of treatment effect.

One hundred and one eligible studies involving 128 pair-wise comparisons were retrieved. Blinding of outcome assessors and incomplete outcome data were frequently judged as 'high' for RoB both for studies with binary and continuous outcome (42.9 and 48.8 per cent, respectively). For binary outcomes, high RoB regarding random sequence generation [odds ratio (OR): 5.97, 95% CI: 2.03, 17.63, P-value: 0.002] and incomplete outcome data (OR: 4.07, 95% CI: 1.03, 16.15, P-value: 0.05) were more likely to provide exaggerated effect estimates.

There is a need for improved clinical trial methodology and reporting, in order to avoid inflated associations and erroneous conclusions.