Comparing masked and traditional visual analysis of multiple-baseline designs

Masked visual analysis (MVA) was developed to complement traditional visual analysis (TVA) and control for Type I error rates. Researchers have empirically tested MVA with generated data and simulated decisions. Our purpose was to evaluate the performance of MVA with real data and human raters. We asked visual analysts who had published at least one single-case research article (n = 36) to evaluate nine multiple-baseline-design-across-participants graphs. Graphs representing different target behaviors were displayed in masked and unmasked presentations. We evaluated the reliability and validity of MVA and TVA. Agreement within each method was similar to that reported in previous studies on visual analysis (MVA ICC = 0.625; TVA ICC = 0.579). Between the two methods, at least 75% of raters' decisions corresponded for six of nine graphs. We discuss the implications of incorporating MVA and future research on analytic methods for single-case experimental designs.