Related Experiment Video
Updated: Dec 26, 2025
10:26
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
4.3K
Multilevel meta-analysis of multiple regression coefficients from single-case experimental studies.
Laleh Jamshidi1,2, Lies Declercq3, Belén Fernández-Castilla3
1Faculty of Psychology and Educational Sciences & ITEC, imec research group at KU Leuven, KU Leuven, University of Leuven, Leuven, Belgium. laleh.jamshidi@uregina.ca.
Behavior Research Methods
|March 8, 2020
Summary
Multilevel meta-analysis models provide robust estimates for treatment effects in single-case experimental studies. Multivariate models offer improved accuracy for overall effects and variance components, even when accounting for dependent effect sizes.
Area of Science:
- Psychology
- Statistics
- Research Methodology
Background:
- Single-case experimental studies (SCES) are crucial for evaluating interventions.
- Meta-analyzing SCES data requires handling dependent effect sizes due to multiple regression coefficients.
- Existing meta-analytic methods may not adequately address the dependence among effect sizes.
Purpose of the Study:
- To compare the performance of different multilevel meta-analytic models in handling dependent effect sizes from SCES.
- To assess the accuracy of overall treatment effect and variance component estimates under various modeling approaches.
- To investigate the robustness of multilevel modeling to misspecified covariance structures.
Main Methods:
- Application of three multilevel meta-analytic models: univariate (ignoring dependence), multivariate (ignoring higher-level covariance), and multivariate (modeling covariance).
- Comparison of model-estimated treatment effects and variance components against true values.
- Evaluation of bias and accuracy under different conditions of true variance and number of studies.
Main Results:
- Multivariate multilevel models yielded better estimates of overall treatment effects and variance components compared to univariate models.
- Model estimates remained robust to misspecified covariance structures at case and study levels.
- Overall treatment effect estimates were unbiased across models, but between-case and between-study variance components showed bias under specific conditions.
Conclusions:
- Multivariate multilevel modeling is recommended for meta-analyzing dependent effect sizes in SCES for improved accuracy.
- While overall treatment effects are reliably estimated, careful consideration of variance component bias is necessary, especially with smaller sample sizes or specific variance structures.
- The findings highlight the importance of appropriate model selection in multilevel meta-analysis of SCES data.
Related Concept Videos
01:25Multiple Regression
3.7K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.7K
01:13Multiple Comparison Tests
4.3K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.3K
02:01Factorial Design
13.6K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
13.6K
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
204
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
204
02:01Group Design
10.0K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
10.0K
01:21Friedman Two-way Analysis of Variance by Ranks
439
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
439