Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Group Design02:01

Group Design

10.1K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
10.1K
Reliability and Validity01:29

Reliability and Validity

13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
Measures of Intelligence01:29

Measures of Intelligence

8.2K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
8.2K
Uncertainty in Measurement: Accuracy and Precision03:37

Uncertainty in Measurement: Accuracy and Precision

99.2K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. 
99.2K
Random and Systematic Errors01:20

Random and Systematic Errors

14.3K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
14.3K
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

14.0K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
14.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

PoTeC: A German naturalistic eye-tracking-while-reading corpus.

Behavior research methods·2025
See all related articles

Related Experiment Video

Updated: Jan 7, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.0K

Replicate Me if You Can: Assessing Measurement Reliability of Individual Differences in Reading Across Measurement

Patrick Haller1, Cui Ding1, Maja Stegenwallner-Schütz2,3

  • 1Department of Computational Linguistics, University of Zurich.

Cognitive Science
|December 30, 2025
PubMed
Summary

Individual differences in sentence processing show unreliable measurement across sessions and methods. Establishing measurement reliability is crucial before studying these individual variations in psycholinguistics.

Keywords:
Eye‐trackingIndividual differencesMeasurement reliabilityNaturalistic reading corpusSelf‐paced readingSentence processingTwo‐task Bayesian modeling

More Related Videos

Using Cholesky Decomposition to Explore Individual Differences in Longitudinal Relations between Reading Skills
06:52

Using Cholesky Decomposition to Explore Individual Differences in Longitudinal Relations between Reading Skills

Published on: September 17, 2019

6.7K
Decomposing the Variance in Reading Comprehension to Reveal the Unique and Common Effects of Language and Decoding
06:33

Decomposing the Variance in Reading Comprehension to Reveal the Unique and Common Effects of Language and Decoding

Published on: October 11, 2018

7.2K

Related Experiment Videos

Last Updated: Jan 7, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.0K
Using Cholesky Decomposition to Explore Individual Differences in Longitudinal Relations between Reading Skills
06:52

Using Cholesky Decomposition to Explore Individual Differences in Longitudinal Relations between Reading Skills

Published on: September 17, 2019

6.7K
Decomposing the Variance in Reading Comprehension to Reveal the Unique and Common Effects of Language and Decoding
06:33

Decomposing the Variance in Reading Comprehension to Reveal the Unique and Common Effects of Language and Decoding

Published on: October 11, 2018

7.2K

Area of Science:

  • Psycholinguistics
  • Cognitive Science
  • Neuroscience

Background:

  • Traditional psycholinguistic theories assume uniform cognitive mechanisms.
  • Recent research highlights the importance of individual differences in human cognition and sentence processing.
  • The Reliability Paradox challenges the assumption that individual-level effects are consistent across sessions and methods.

Purpose of the Study:

  • To assess the measurement reliability of individual differences in sentence processing.
  • To investigate the consistency of effects across multiple sessions and methods (eye-tracking, self-paced reading).
  • To examine reliability for various psycholinguistic predictors: word length, lexical frequency, surprisal, dependency length, and integration load.

Main Methods:

  • Collected a naturalistic eye movement corpus with four sessions per participant (two eye-tracking, two self-paced reading).
  • Employed a two-task Bayesian hierarchical model to analyze measurement reliability.
  • Evaluated individual-level effects for established psycholinguistic phenomena.

Main Results:

  • High reliability across sessions for word length effects.
  • Moderate reliability for lexical frequency, dependency distance, and integration load.
  • Low reliability for surprisal.
  • Low to moderate cross-method reliability for most predictors, and poor reliability for syntactic integration predictors.

Conclusions:

  • Measurement reliability varies significantly across different psycholinguistic predictors.
  • Reliability is particularly low for higher-level cognitive and syntactic phenomena.
  • Establishing measurement reliability is a critical prerequisite for valid inferences about individual differences in sentence processing.