Characterizing phonemic fluency by transfer learning with deep language models

Area of Science:

• Neuroscience
• Psycholinguistics
• Computational Linguistics

Background:

• Phonemic fluency tasks traditionally focus on response quantity, overlooking qualitative word choice patterns.
• Underlying neurological disorders can influence both correct and erroneous word selection in fluency tasks.
• Previous analyses have not comprehensively examined qualitative features of phonemic fluency performance in relation to brain lesions.

Purpose of the Study:

• To conduct the first comprehensive qualitative analysis of correct and incorrect words generated during the phonemic ('S') fluency test.
• To investigate the utility of large language models and stochastic block modelling for analyzing word sequences in phonemic fluency.
• To determine if deep language representations can improve the detection of frontal lesions using phonemic fluency data.

Main Methods:

• Qualitative analysis of single words generated in the phonemic fluency task, categorizing errors, low-frequency words, and clustering/switching.
• Stochastic block modelling of Generative Pretrained Transformer 3 (GPT-3)-based deep language representations of word sequences.
• Predictive modelling to assess the accuracy of detecting frontal lesions using GPT-3-derived representations versus native features.

Main Results:

• Qualitative analysis revealed lesion-specific patterns: non-lateralized frontal effects for profanities, left frontal effects for proper nouns/permutations, left posterior effects for perseverations, and left frontal effects for low-frequency correct words.
• Large language model analysis identified five distinct communities with unique word selection patterns linked to demographic and clinical features.
• Predictive models using GPT-3 representations demonstrated higher fidelity in predicting frontal lesions compared to models using native features.

Conclusions:

• Phonemic fluency performance in patients with frontal lesions exhibits characteristic qualitative patterns.
• Characterizing qualitative features of phonemic fluency using large language models and stochastic block modelling offers significant inferential and diagnostic value.
• This approach enhances the potential of fluency tasks for detecting and understanding the impact of neurological conditions like frontal lesions.