Related Experiment Video
Updated: Sep 28, 2025
08:12
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
2.6K
Statistical Decision-Making Accuracies for Some Overlap- and Distance-based Measures for Single-Case Experimental
Michael T Carlin1, Mack S Costello1
1Department of Psychology, Rider University, 2083 Lawrenceville Road, Lawrenceville, NJ 08648 USA.
Perspectives on Behavior Science
|March 28, 2022
Summary
Choosing the best quantitative measure for single-case experimental designs (SCEDs) is crucial. Tau, RD, and g measures offer the highest accuracy for treatment effect decisions in SCEDs.
Area of Science:
- Behavioral Science
- Research Methodology
- Psychometrics
Background:
- Selecting appropriate quantitative measures for single-case experimental designs (SCEDs) is complex due to numerous available metrics.
- Existing measures, including overlap-based and distance-based metrics, have faced valid criticisms regarding their efficacy.
Purpose of the Study:
- To compare the performance of various quantitative measures used in SCEDs.
- To evaluate Type I error rates and statistical power across different SCED parameters.
- To identify the most accurate measures for decision-making in SCED research.
Main Methods:
- Comparison of overlap-based measures (e.g., percentage nonoverlapping data) and distance-based measures (e.g., Cohen's d).
- Evaluation across diverse SCED scenarios with equal phase observations (3-10).
- Assessment of Type I error rate and statistical power for each measure.
Main Results:
- Tau and distance-based measures (RD and g) demonstrated superior decision accuracy.
- Overlap-based measures like percentage nonoverlapping data and the dual-criterion method showed lower performance.
- Tau excelled in identifying the presence or absence of treatment effects.
Conclusions:
- Tau is recommended for determining the existence of treatment effects in SCEDs.
- RD or g are advised for quantifying the magnitude of treatment effects.
- The study provides evidence-based recommendations for selecting quantitative measures in SCED research.
Related Concept Videos
01:11Statistical Analysis: Overview
8.3K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
8.3K
01:13Accuracy and Errors in Hypothesis Testing
347
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
347
01:19Testing a Claim about Standard Deviation
2.5K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.5K
01:04Significance Testing: Overview
4.2K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
4.2K
01:00Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
01:15Study Design in Statistics
9.4K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
9.4K