Related Experiment Video
Updated: May 9, 2026
10:26
Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
Analysis of single-case data: randomisation tests for measures of effect size
Mieke Heyvaert1, Patrick Onghena
1a Faculty of Psychology and Educational Sciences , University of Leuven , Belgium.
Neuropsychological Rehabilitation
|July 20, 2013
Summary
Randomizing single-case experiments enhances validity. This study details randomized designs and introduces an effect size index for robust intervention effect analysis in single-entity studies.
Area of Science:
- Behavioral Science
- Research Methodology
- Psychology
Background:
- Single-case experiments are crucial for evaluating interventions in individual cases.
- Enhancing internal and statistical conclusion validity is paramount for reliable findings.
- Randomization offers a powerful tool to strengthen the rigor of single-case designs.
Purpose of the Study:
- To explain the design and analysis of randomized single-case experiments.
- To introduce randomisation tests for various randomized single-case designs.
- To propose using an effect size index within randomisation tests to quantify intervention effects.
Main Methods:
- Detailed explanation of designing randomized single-case phase and alternation studies.
- Guidance on constructing randomized simultaneous and sequential replication studies.
- Methodology for conducting randomisation tests, including the use of an effect size index.
Main Results:
- Randomisation significantly improves the validity of single-case experiments.
- Randomisation tests provide a robust statistical framework for analyzing these designs.
- Incorporating an effect size index allows for the quantification of intervention magnitude.
Conclusions:
- Randomized single-case experimental designs offer superior internal and statistical conclusion validity.
- Randomisation tests, especially when combined with effect size indices, provide comprehensive analysis of intervention effects.
- The proposed methods are illustrated with an ABAB phase study and available free software.
Related Concept Videos
01:13Randomized Experiments
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
01:17Sign Test for Matched Pairs
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in value between...
To conduct the sign test, we first calculate the differences in value between...
01:37Statistical Significance
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
01:15One-Way ANOVA: Equal Sample Sizes
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
01:15One-Way ANOVA: Unequal Sample Sizes
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
01:21Friedman Two-way Analysis of Variance by Ranks
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...