Predicting speech from a cortical hierarchy of event-based time scales

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Linguistics

Background:

• Understanding how the brain processes sequential information and context is crucial for explaining predictive language comprehension.
• Previous models often struggle to capture the dynamic and hierarchical nature of contextual integration in real-time.
• The temporal unfolding of context and its impact on neural predictions remain key research questions.

Purpose of the Study:

• To investigate the neural coding scheme for incorporating temporal context in predictive language comprehension.
• To explore how the brain represents and updates contextual information at different time scales.
• To elucidate the hierarchical organization of predictive processing in the brain.

Main Methods:

• Training artificial neural networks to predict the next word across five stacked time scales.
• Utilizing model-based functional magnetic resonance imaging (mb-fMRI) to observe brain activity.
• Analyzing the emergence of an event-based 'surprisal hierarchy' along a temporoparietal pathway.

Main Results:

• Evidence for a neural coding scheme that sparsely updates context at event boundaries.
• Observation of an event-based 'surprisal hierarchy' along the temporoparietal pathway.
• Demonstration that surprisal at different time scales modulates neural connectivity.
• Finding that continuously updated context primarily affects short time scales in temporoparietal activity.

Conclusions:

• The brain employs a hierarchical, multilayered organization for predictive language comprehension.
• Context is represented through increasingly coarse events, enabling computationally efficient and contextually diverse predictions.
• Event-based sparse updating of context is a key mechanism for predictive processing.