EMTeC: A corpus of eye movements on machine-generated texts

Area of Science:

• Cognitive Science
• Computational Linguistics
• Human-Computer Interaction

Background:

• Understanding human reading behavior is crucial for AI development.
• Existing corpora lack naturalistic eye-tracking data on machine-generated text.

Purpose of the Study:

• Introduce the Eye movements on Machine-generated Texts Corpus (EMTeC).
• Provide a resource for studying human reading of AI-generated content.
• Facilitate research on AI model interpretability and reading behavior.

Main Methods:

• Collected eye-tracking data from 107 participants reading texts generated by three large language models.
• Utilized five decoding strategies and six text-type categories.
• Included pre-processed eye movement data, model internals, and linguistic annotations.

Main Results:

• The EMTeC corpus offers raw and processed eye-tracking data, including fixation sequences and reading measures.
• It contains language model internals (transition/attention scores, hidden states) and linguistic annotations.
• The corpus includes corrected fixation sequences accounting for vertical calibration drift.

Conclusions:

• EMTeC supports diverse research, including reading behavior analysis on AI text and decoding strategies.
• It aids in developing new data processing algorithms and enhancing AI model interpretability.
• The corpus is valuable for assessing predictive models of human reading times.