Semantic language decoding across participants and stimulus modalities.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Linguistics

Background:

• Brain decoders reconstruct language from semantic representations to aid communication.
• Current methods require extensive participant-specific training data.
• Impaired language production often co-occurs with impaired comprehension.

Purpose of the Study:

• To test language decoding without participant-specific linguistic training data.
• To investigate cross-participant semantic decoder transfer using functional alignment.
• To assess the robustness of cross-participant decoders to brain lesions.

Main Methods:

• Trained semantic decoders on reference participant brain responses.
• Used functional alignment to transfer decoders to a goal participant.
• Compared functional alignment using linguistic (story) and non-linguistic (movie) stimuli.
• Assessed decoder robustness by excluding brain regions.

Main Results:

• Cross-participant decoders achieved semantically related predictions without goal participant linguistic data.
• Functional alignment using movies was effective for decoder transfer.
• Semantic representation sharing between language and vision was comparable across cortical regions.
• Cross-participant decoders were robust to the exclusion of single brain regions.

Conclusions:

• Cross-participant decoding significantly reduces the need for individual linguistic training.
• This approach may enable language decoding for individuals with severe language impairments.
• Brain semantic representations are largely shared across individuals and modalities.