Accuracy and Precision of an Effect Size and Its Variance From a Multilevel Model for Cluster Randomized Trials: A...

Area of Science:

• Biostatistics
• Statistical Modeling
• Clinical Trials

Background:

• Cluster randomized trials (CRTs) are essential for evaluating interventions in group-level settings.
• Accurate effect size estimation is crucial for interpreting CRT results.
• Existing effect size measures may exhibit bias, particularly with small sample sizes or unequal cluster sizes.

Purpose of the Study:

• To investigate the accuracy and precision of a novel effect size (MLM ES) derived from multilevel modeling (MLM) for CRTs.
• To compare the performance of MLM ES with Hedges' dB, an effect size based on summary statistics.
• To evaluate the behavior of MLM ES under various conditions, including different sample sizes and cluster sizes.

Main Methods:

• Monte Carlo simulation methods were employed to assess the accuracy and precision of MLM ES.
• MLM ES was compared against Hedges' dB across simulated scenarios.
• The study analyzed performance concerning bias, efficiency, and consistency.

Main Results:

• MLM ES demonstrated acceptable accuracy, efficiency, and consistency across all simulated conditions.
• MLM ES showed less negative bias than Hedges' dB with small sample sizes, avoiding overestimation of between-cluster variance.
• Both MLM ES and Hedges' dB exhibited bias when cluster sizes were unequal, though MLM ES performed comparably to Hedges' dB with large sample sizes.

Conclusions:

• MLM ES is a viable and often preferable effect size measure for CRTs, particularly when dealing with small sample sizes.
• The proposed MLM ES offers improved accuracy and reduced bias compared to Hedges' dB in specific scenarios.
• Further consideration of unequal cluster sizes is warranted for both effect size measures.