Meta-analysis, Simpson's paradox, and the number needed to treat
Related Experiment Video
Updated: Sep 7, 2026

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI)
Published on: November 27, 2019
Douglas G Altman1, Jonathan J Deeks
1Centre for Statistics in Medicine, Institute of Health Sciences, Old Road, Headington, Oxford, UK. doug.altman@cancer.org.uk
Background:
There is debate concerning methods for calculating numbers needed to treat (NNT) from results of systematic reviews.
Methods:
We investigate the susceptibility to bias for alternative methods for calculating NNTs through illustrative examples and mathematical theory.
Results:
Two competing methods have been recommended: one method involves calculating the NNT from meta-analytical estimates, the other by treating the data as if it all arose from a single trial. The 'treat-as-one-trial' method was found to be susceptible to bias when there were imbalances between groups within one or more trials in the meta-analysis (Simpson's paradox). Calculation of NNTs from meta-analytical estimates is not prone to the same bias. The method of calculating the NNT from a meta-analysis depends on the treatment effect used. When relative measures of treatment effect are used the estimates of NNTs can be tailored to the level of baseline risk.
Conclusions:
The treat-as-one-trial method of calculating numbers needed to treat should not be used as it is prone to bias. Analysts should always report the method they use to compute estimates to enable readers to judge whether it is appropriate.
Related Concept Videos
Regression Toward the Mean
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Sign Test for Matched Pairs
To conduct the sign test, we first calculate the differences in value between...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Comparing the Survival Analysis of Two or More Groups
Hazard Ratio
For example, in a clinical trial evaluating a...
