Comparison of Large Language Model with Aphasia

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Neurolinguistics

Background:

• Large language models (LLMs) display fluent but often inaccurate responses, mirroring human aphasia.
• The internal information processing mechanisms of LLMs and aphasic brains remain largely unexplored.
• Behavioral similarities prompt investigation into potential parallels in network dynamics.

Purpose of the Study:

• To compare the network dynamics of LLMs (ALBERT, GPT-2, Llama-3.1) with human aphasic brain activity.
• To determine if energy landscape analysis can differentiate between types of aphasia and LLM network states.
• To explore the potential of this analysis as a diagnostic and improvement tool for LLMs.

Main Methods:

• Applied energy landscape analysis to quantify network dynamics in LLMs and aphasic brains.
• Measured transition frequency (state-to-state movement) and dwelling time (time spent in a state).
• Analyzed the frequency spectrums of these indices to identify distinct patterns.

Main Results:

• Network dynamics in LLMs showed highly polarized distributions for transition frequency and dwelling time.
• Receptive aphasia exhibited bimodal distributions for both indices, while expressive aphasia showed uniform distributions.
• The polarization of transition frequency and dwelling time accurately classified receptive aphasia, expressive aphasia, and controls.

Conclusions:

• LLMs demonstrate internal information processing similarities to receptive aphasia.
• Energy landscape analysis provides a novel method for diagnosing and classifying aphasia.
• This approach offers a potential tool for improving LLM performance by identifying processing similarities to healthy or impaired human cognition.