Perceiving and remembering speech depend on multifractal nonlinearity in movements producing and exploring speech

Area of Science:

• Cognitive Science
• Linguistics
• Physics

Background:

• Speech perception and memory rely on active cognitive processes.
• Existing gestural theories primarily focus on speaker movements, neglecting listener contributions.
• Communication is often viewed as a constraint on movement, not a driver of perception.

Purpose of the Study:

• To investigate the role of listener movement in speech perception and memory.
• To apply multifractal geometry of physical movement to analyze speaker-listener interactions.
• To explore the impact of self-paced listening on narrative comprehension and word recognition.

Main Methods:

• Participants controlled the pace of auditory narrative presentation (self-paced listening).
• Multifractal geometry was used to analyze the nonlinear dynamics of listener movements.
• Speech perception fluency, narrative comprehension, and word recognition were assessed.

Main Results:

• Multifractal analysis of listener movements predicted speech perception fluency.
• Nonlinear dynamics in self-paced listening movements supported memory for spoken language.
• Listener movement complexity correlated with narrative comprehension and word recall.

Conclusions:

• Multifractal geometry provides a unified framework for understanding speaker and listener movements in speech.
• Listener-controlled speech exploration significantly enhances memory for perceived spoken narratives.
• Findings suggest calibrating multifractal structure in text-to-speech synthesis for improved user experience.