Contextual assembly of lexical functions in large language models.

Area of Science:

• Psycholinguistics
• Computational Linguistics
• Cognitive Science

Background:

• Neural network models have been crucial for psycholinguistic research on lexical processing.
• Large language models (LLMs) present a novel computational approach to investigate the mental lexicon.

Purpose of the Study:

• To evaluate the capacity of LLMs to generate psycholinguistic ratings of words.
• To compare LLM-generated ratings with human judgments across different experimental contexts.
• To assess LLM performance in predicting word naming latencies.

Main Methods:

• Four LLMs were utilized across three experiments to generate word ratings.
• Contextual manipulations were applied to LLM prompts to assess calibration effects.
• LLM-generated word naming latencies were compared against human data.

Main Results:

• LLM ratings demonstrated high correlations with human ratings, influenced by rating ambiguity.
• Contextual variations in LLM prompts improved the calibration and correlation with human data.
• LLM inter-rater variability mirrored human inter-rater variability.
• LLMs exhibited functional deviations in generating word naming latencies compared to humans.

Conclusions:

• LLMs leverage context to assemble generalized lexical functions, rather than retrieving stored data.
• Current LLM lexical function assembly is constrained by co-occurrence patterns in training data.
• Future LLM development requires finer-grained temporal patterns for modeling online lexical processes.