Mirror manifolds: partially overlapping neural subspaces for speaking and listening

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Linguistics

Background:

• Conversational participants must link words to speakers while retaining general word meanings.
• Existing theories inadequately address the challenge of differentiating speakers and generalizing word semantics.

Purpose of the Study:

• To investigate how the brain resolves the speaker generalization/differentiation dilemma.
• To test the hypothesis of a vectorial semantic code involving collinear and orthogonal subspaces.

Main Methods:

• Recorded hippocampal single neurons in epilepsy patients during conversations.
• Analyzed semantic encoding for spoken and heard words.
• Examined coding principles in three-person conversations.

Main Results:

• Found clear semantic encoding for both word production and reception in the hippocampus.
• Hippocampal neural codes for word meaning were a blend of orthogonal and collinear coding.
• Semantic categories like body parts and names showed greater differentiation between speakers than function words and verbs.
• The hippocampus orthogonalized coding more between self and other than between two others.

Conclusions:

• The hippocampus employs a vectorial semantic code to bind word meanings with speaker identity.
• This coding strategy offers a novel solution to a fundamental challenge in natural language processing.
• Findings advance our understanding of neural mechanisms underlying social communication.