Related Experiment Video
Updated: Jun 22, 2026
06:55
Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Empirical vs natural weighting in random effects meta-analysis
1Department of Epidemiology and Health Policy Research, College of Medicine, University of Florida, PO Box 100177, Gainesville, FL 32610-0177, USA. jshuster@biostat.ufl.edu
Statistics in Medicine
|May 29, 2009
Summary
Empirically weighted random effects meta-analysis may be biased because sample sizes are treated as fixed. New methods are proposed to address this bias, suggesting a reevaluation of past studies using this approach.
Area of Science:
- Biostatistics
- Medical Research Methodology
Background:
- Random effects meta-analysis is widely used to synthesize evidence.
- Empirically based weighting methods, like DerSimonian-Laird, are popular but rely on potentially flawed assumptions regarding sample sizes.
Purpose of the Study:
- To critically evaluate the validity of empirically based weighting in random effects meta-analysis.
- To propose and demonstrate alternative methods for random effects meta-analysis that address bias related to sample size treatment.
Main Methods:
- Demonstration of bias in empirical weighting by treating sample sizes as non-random.
- Proposal of two alternative methods: 1) estimating the arithmetic mean of population effect sizes using an unweighted approach, and 2) estimating patient-level effect sizes weighted by total sample size.
- Application of methods to a meta-analysis of a nasal decongestant.
Main Results:
- Empirical weighting, including the DerSimonian-Laird method, risks substantial bias.
- An unweighted mean of study effect sizes is recommended for estimating the arithmetic mean of population effect sizes.
- Weighting study effect sizes by total sample size is necessary for consistent estimation of patient-level effect sizes.
- The proposed methods yield counter-intuitive results compared to the DerSimonian-Laird approach.
Conclusions:
- Past publications using empirically weighted random effects meta-analysis should be revisited.
- Empirically based weighted random effects meta-analysis should be used with extreme caution or avoided in favor of proposed methods.
- Future analyses should include secondary analyses based on the principles presented to supplement primary findings.
Related Concept Videos
00:57Weighted Mean
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:13Randomized Experiments
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
03:37Uncertainty in Measurement: Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
02:30Naturalistic Observations
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...