Parallel hierarchical encoding of linguistic representations in the human auditory cortex and recurrent automatic...

Area of Science:

• Neuroscience
• Computational Linguistics
• Artificial Intelligence

Background:

• Human speech processing transforms acoustic signals into linguistic representations, inspiring automatic speech recognition (ASR).
• Previous ASR-brain comparisons were limited by biologically implausible models, narrow features, and focus on predictability over shared representations.
• Studies comparing brains to text-based models neglect crucial acoustic processing stages.

Purpose of the Study:

• To investigate shared hierarchical representations between the human brain and recurrent ASR models during speech processing.
• To bridge gaps in prior research by using high-resolution intracranial recordings and advanced ASR models.
• To explore how both systems encode linguistic features from acoustic to semantic levels.

Main Methods:

• Utilized high-resolution intracranial electroencephalography (iEEG) recordings from human participants.
• Employed a recurrent artificial intelligence model for automatic speech recognition.
• Compared neural activity patterns with representations within corresponding layers of the ASR model.

Main Results:

• Demonstrated a striking correspondence in the hierarchical encoding of linguistic features between the brain and the ASR model.
• Showed alignment between neural activity in auditory cortex regions and specific ASR model layers.
• Confirmed that both systems encode similar features across processing stages: acoustic, phonetic, lexical, and semantic.

Conclusions:

• The human brain and ASR systems converge on similar strategies for speech processing despite architectural differences.
• Findings offer insights into optimal computational principles for linguistic representation.
• Highlights shared constraints shaping both human and artificial speech processing.