Comparison of Large Language Model with Aphasia

Large language models (LLMs) respond fluently but often inaccurately, which resembles aphasia in humans. Does this behavioral similarity indicate any resemblance in internal information processing between LLMs and aphasic humans? Here, we address this question by comparing the network dynamics between LLMs-ALBERT, GPT-2, Llama-3.1 and one Japanese variant of Llama-and various aphasic brains. Using energy landscape analysis, we quantify how frequently the network activity pattern is likely to move from one state to another (transition frequency) and how long it tends to dwell in each state (dwelling time). First, by investigating the frequency spectrums of these two indices for brain dynamics, we find that the degrees of the polarization of the transition frequency and dwelling time enable accurate classification of receptive aphasia, expressive aphasia and controls: receptive aphasia shows the bimodal distributions for both indices, whereas expressive aphasia exhibits the most uniform distributions. In parallel, we identify highly polarized distributions in both transition frequency and dwelling time in the network dynamics in the four LLMs. These findings indicate the similarity in internal information processing between LLMs and receptive aphasia, and the current approach can provide a novel diagnosis and classification tool for LLMs and help their performance improve.