Related Experiment Video
Updated: Apr 14, 2026
07:34
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
18.1K
Interrater agreement between visual analysts of single-case data: a meta-analysis.
Jennifer Ninci1, Kimberly J Vannest2, Victor Willson2
1Texas A&M University, College Station, USA jninci@tamu.edu.
Behavior Modification
|April 17, 2015
Summary
Visual analysis of single-case research data shows moderate agreement. Factors like rater expertise and clear definitions can improve reliability in behavior change research.
Area of Science:
- Behavioral Science
- Research Methodology
Background:
- Visual analysis is a primary method for interpreting single-case research data.
- Previous research indicates low interrater agreement in visual analysis of graphed data.
Purpose of the Study:
- To conduct a meta-analysis on interrater agreement in visual analysis of single-case research.
- To identify moderators influencing agreement among visual analysts.
Main Methods:
- A meta-analysis of 19 peer-reviewed articles comprising 32 effects.
- Evaluation of moderators including design families, rater expertise, contextual information, visual aids, operational definitions, and scale ranges.
Main Results:
- The overall weighted interrater agreement proportion was .76.
- Identified moderator variables demonstrated low to adequate levels of interrater agreement.
Conclusions:
- Interrater agreement in visual analysis of single-case data is moderate.
- Specific factors influence agreement, suggesting areas for methodological improvement in behavior change research.
Related Concept Videos
01:18One-Way ANOVA
15.1K
One-way ANOVA analyzes more than three samples categorized by one factor. For example, it can compare the average mileage of sports bikes. Here, the data is categorized by one factor - the company. However, one-way ANOVA cannot be used to simultaneously compare the sample mean of three or more samples categorized by two factors. An example of two factors would be sports bikes from different companies driven in different terrains, such as a desert or snowy landscape. Here, two-way ANOVA is used...
15.1K
01:17Two-Way ANOVA
3.7K
The two-way ANOVA is an extension of the one-way ANOVA. It is a statistical test performed on three or more samples categorized by two factors - a row factor and a column factor. Ronald Fischer mentioned it in 1925 in his book 'Statistical Methods for Researchers.'
The two-way ANOVA analysis initially begins by stating the null hypothesis that there is an interaction effect between the two factors of a dataset. This effect can be visualized using line segments formed by joining the...
The two-way ANOVA analysis initially begins by stating the null hypothesis that there is an interaction effect between the two factors of a dataset. This effect can be visualized using line segments formed by joining the...
3.7K
01:15One-Way ANOVA: Equal Sample Sizes
4.5K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.5K
01:17Sign Test for Matched Pairs
477
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
477
01:15One-Way ANOVA: Unequal Sample Sizes
7.1K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
7.1K
01:09Wilcoxon Signed-Ranks Test for Matched Pairs
624
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
624