Effects of serial dependency on the agreement between visual and statistical inference

Area of Science:

• Behavioral science
• Data analysis
• Psychometrics

Background:

• Visual analysis and time-series analysis are common methods for interpreting behavioral data.
• Serial dependency in scores can affect the agreement between these analytical approaches.
• Previous recommendations suggest using time-series analysis to supplement visual analysis.

Purpose of the Study:

• To investigate the conditions under which visual and time-series inferences from behavioral data disagree.
• To identify factors that disrupt the agreement between these two analytical methods.

Main Methods:

• Comparison of inferences derived from visual inspection of behavioral data.
• Application of time-series analysis to the same behavioral datasets.
• Statistical analysis to quantify agreement and disagreement between the two methods.

Main Results:

• Serial dependency in behavioral scores significantly disrupts agreement between visual and time-series analyses.
• Disagreement is most pronounced when data exhibit high levels of autocorrelation.
• Time-series analysis may indicate reliable behavioral changes when visual analysis does not, particularly with autocorrelated data.

Conclusions:

• Researchers should be cautious when combining visual and time-series analyses of behavioral data.
• High autocorrelation in data is a key factor leading to discrepancies between the two methods.
• The findings challenge the straightforward application of time-series analysis to confirm visual interpretations without considering data characteristics.