The human language processing system straightens natural speech.

Area of Science:

• Neuroscience
• Computational Linguistics
• Cognitive Science

Background:

• Large language models excel at next-word prediction, hinting at temporal prediction's importance in language.
• The impact of temporal prediction on human speech representation structure is not well understood.

Purpose of the Study:

• To investigate if temporal prediction facilitates speech processing through representational trajectory straightening.
• To explore the relationship between neural timescales and representational geometry in speech processing.

Main Methods:

• Developed a novel fMRI-based methodology to measure the curvature of neural representational trajectories.
• Utilized a connection between single-unit response timescales and population trajectory curvature.
• Analyzed brain responses to natural and perturbed speech stimuli in humans and a speech representation model (wavLM).

Main Results:

• Speech representational trajectories showed increasing straightening along the cortical hierarchy, from curved in auditory areas to straight in higher-level regions.
• Hierarchical straightening effects were more pronounced for stimuli with natural statistical structures.
• The wavLM model, aligned with human brain responses, also exhibited these hierarchical straightening effects.

Conclusions:

• Temporal prediction is directly linked to the geometric properties of neural speech representations.
• The cortical hierarchy organizes speech representations with timescales that facilitate prediction.
• This research bridges the gap between computational goals (prediction) and neural implementation (representational geometry).