Automating Intended Target Identification for Paraphasias in Discourse Using a Large Language Model

Area of Science:

• Computational linguistics
• Neuroscience
• Speech-language pathology

Background:

• Paraphasias are common in aphasia, but automated tools for their identification and classification are lacking.
• Accurate analysis of paraphasias is crucial for understanding and treating aphasia.

Purpose of the Study:

• To fine-tune a large language model (LLM) for automatic prediction of paraphasia targets in discourse.
• To develop an automated tool for the identification and classification of paraphasias in people with aphasia (PWA).

Main Methods:

• Utilized 332 Cinderella story retellings from PWA with 2,489 identified paraphasias.
• Supplemented training data with 256 control sessions and 2,415 synthetic paraphasias.
• Fine-tuned an LLM to predict paraphasia targets within the story retelling context.

Main Results:

• The LLM achieved 50.7% accuracy in exactly matching human-identified paraphasia targets.
• Performance was comparable whether fine-tuned on PWA data alone or with control data.
• The model showed improved accuracy for less ambiguous targets and participants with milder aphasia.

Conclusions:

• Demonstrated the feasibility of automatically identifying paraphasia targets using surrounding language context.
• This work represents a step towards automated aphasic discourse analysis.
• Future research will incorporate phonological information to enhance predictive accuracy.