Cognitive plausibility of count-based versus prediction-based word embeddings: A large-scale N400 study

The N400 is a central electrophysiological event-related-potential (ERP) marker thought to reflect meaning comprehension in the human brain. Typically, the N400 is larger when a word does not fit into a specific context (e.g., I drink coffee with cream and dog). Thus, one core factor determining the N400 amplitude is thought to be the predictability of a word within its context. Here, both long-term memory associations and short-term discourse context influence the N400 amplitude. In the present study, we used the N400 as a marker to investigate the cognitive plausibility of semantic similarity measures. Specifically, we compared traditional count-based measures to modern machine learning tools such as prediction-based word embeddings to assess whether prediction-based techniques potentially encapsulate learning mechanisms that align more closely with psychological plausibility. To do so, we examined the relationship between different similarity measures (LSA, HAL and word2vec) and the N400 amplitude in a large scale re-analysis of previously published EEG data. Model comparison suggested a superiority of HAL over LSA as a predictor in explaining single-trial N400 amplitudes, and also a benefit of prediction-based methods over count-based methods. This result aligns with the notion that such models might in the future provide further insights into how the brain navigates language understanding.