Reproducibility in Small-N Treatment Research: A Tutorial Using Examples From Aphasiology

Area of Science:

• Communication Sciences and Disorders
• Aphasia Research
• Evidence-Based Interventions

Background:

• Small-N studies are crucial for evidence-based interventions in communication science and disorders, particularly for aphasia treatments.
• Lack of guidance on reproducible analyses and effect size selection in small-N studies hinders scientific review, rigor, and replication.
• Understanding effect sizes is vital for interpreting and synthesizing research findings in aphasiology.

Purpose of the Study:

• To demonstrate reproducible analysis methods for small-N studies using common effect sizes in aphasia research.
• To provide a conceptual understanding of various effect sizes relevant to communication disorders research.
• To enhance the rigor and replicability of small-N treatment research in aphasiology.

Main Methods:

• Tutorial utilizing the statistical programming language R with published data from Wambaugh et al. (2017).
• Demonstration of reproducible analysis for within-case standardized mean difference, proportion of maximal gain, tau-U, and mixed-effects models (frequentist and Bayesian).
• Discussion of strengths, weaknesses, reporting requirements, and design impacts on effect sizes.

Main Results:

• Reproducible code provided for calculating and comparing multiple effect sizes.
• Implementation of frequentist and Bayesian mixed-effects models for small-N data analysis.
• Availability of data, code, and an interactive web application for researchers, clinicians, and students.

Conclusions:

• Reproducible research practices are essential for transparency in small-N treatment studies.
• Informed selection and interpretation of effect sizes improve the quality of small-N research.
• Commitment to reproducibility and understanding effect sizes can advance the evidence base for clinical services in communication sciences and disorders.