Detecting false positives in multielement designs: implications for brief assessments.

Area of Science:

• Behavior analysis
• Research methodology

Background:

• Multielement designs are common in behavior analysis for evaluating interventions.
• Concerns exist regarding the potential for false positives in these designs, particularly with brief assessments.
• Continuous duration recording (CDR) and interval recording are frequently used measurement methods.

Purpose of the Study:

• To assess the rate of false positives in multielement designs across different recording methods and interval sizes.
• To investigate the impact of varying data paths and data points per path on false positive rates.
• To evaluate the effectiveness of newly proposed rules in mitigating false positives.

Main Methods:

• Generated 6,000 simulated graphs of multielement designs with varied data paths and points per path.
• Utilized continuous duration recording (CDR) and interval recording (10-s and 1-min intervals).
• Visually analyzed graphs for false positives (Experiment 1) and applied new rules to reduce them (Experiment 2).

Main Results:

• Graphs with only two sessions per condition showed the highest false positive rates for all recording methods.
• Graphs with four or five sessions per condition demonstrated the lowest false positive rates.
• Newly introduced rules reduced false positives to below 5% for most graphs, except those with minimal data (two paths, two points).

Conclusions:

• The number of sessions per condition critically influences the accuracy of multielement designs.
• Brief assessments with few data points increase the risk of false positives.
• The proposed rules show promise for enhancing the validity of brief experimental analyses.