iMAP and iMAP2 produce erroneous statistical maps of eye-movement differences

Area of Science:

• Cognitive Psychology
• Behavioral Research Methods
• Data Analysis

Background:

• The iMAP and iMAP2 programs offer a method for analyzing eye-movement data, including heat maps and significance tests.
• These tests aim to account for spatial autocorrelation in fixation locations for group comparisons.

Purpose of the Study:

• To critically evaluate the statistical significance tests implemented in the iMAP and iMAP2 programs.
• To identify potential errors in the algorithms used for analyzing eye-movement data and group differences.

Main Methods:

• The study re-evaluated the significance tests in iMAP/iMAP2 using a conventional randomization test.
• A modified statistical technique, incorporating a t-test, error functions, and spatial autocorrelation, was developed and applied.
• Simulated data with and without true differences were analyzed using both iMAP/iMAP2 and the modified method.

Main Results:

• The iMAP/iMAP2 significance tests produced erroneous results, claiming significant differences even with random group allocation.
• The modified statistical technique found no significant differences between example participant groups.
• Simulations showed iMAP/iMAP2 consistently found significant differences, while the modified method only did for larger sample sizes.

Conclusions:

• The statistical algorithms in iMAP/iMAP2 are inappropriate, leading to potentially erroneous conclusions in eye-movement research.
• Previous studies relying on iMAP/iMAP2 for group comparisons of fixation patterns may need re-evaluation.
• A modified statistical approach offers a more reliable method for analyzing eye-movement data with spatial autocorrelation.