Assessing naming errors using an automated machine learning approach

After left hemisphere stroke, 20%-50% of people experience language deficits, including difficulties in naming. Naming errors that are semantically related to the intended target (e.g., producing "violin" for picture HARP) indicate a potential impairment in accessing knowledge of word forms and their meanings. Understanding the cause of naming impairments is crucial to better modeling of language production as well as for tailoring individualized rehabilitation. However, evaluation of naming errors is typically by subjective and laborious dichotomous classification. As a result, these evaluations do not capture the degree of semantic similarity and are susceptible to lower interrater reliability because of subjectivity.

We investigated whether a computational linguistic measure using word2vec (Mikolov, Chen, et al., 2013) addressed these limitations by evaluating errors during object naming in a group of patients during the acute stage of a left-hemisphere stroke (N = 105).

Pearson correlations demonstrated excellent convergent validity of word2vec's semantically related estimates of naming errors and independent tests of access to lexical-semantic knowledge (p < .0001). Further, multiple regression analysis showed word2vec's semantically related estimates were significantly better than human error classification at predicting performance on tests of lexical-semantic knowledge.

Useful to both theorists and clinicians, our word2vec-based method provides an automated, continuous, and objective psychometric measure of access to lexical-semantic knowledge during naming. (PsycInfo Database Record (c) 2022 APA, all rights reserved).