From a single-level analysis to a multilevel analysis of single-case experimental designs

Area of Science:

• Psychology
• Statistics
• Research Methodology

Background:

• Single-case experimental designs (SCEDs) generate valuable data for evaluating interventions.
• Synthesizing results from multiple SCEDs presents analytical challenges.
• Existing methods for aggregating SCED data may not fully capture hierarchical structures.

Purpose of the Study:

• To present and elaborate on multilevel modeling approaches for synthesizing single-case experimental design data.
• To introduce two-level and three-level multilevel models for summarizing SCED results across cases and studies.
• To investigate the sensitivity of treatment effect estimates to different modeling specifications and assumptions.

Main Methods:

• Application of basic and alternative multilevel models (two-level and three-level) to real-world datasets.
• Systematic evaluation of how varying model specifications and assumptions impact estimated treatment effects.
• Comparative analysis of results obtained from different plausible multilevel modeling approaches.

Main Results:

• Treatment effect estimates can be dependent on the chosen multilevel model specifications and underlying assumptions.
• The robustness of conclusions can be assessed by comparing results across a range of plausible models.
• Sensitivity analyses are crucial for understanding the reliability of findings derived from SCED data synthesis.

Conclusions:

• Researchers should not rely on a single multilevel model for synthesizing SCED data.
• Conducting multiple plausible multilevel analyses is recommended to assess the stability of conclusions.
• Enhanced confidence in findings is achieved when results remain consistent across various modeling options and assumptions.