How useful are corpus-based methods for extrapolating psycholinguistic variables?

Area of Science:

• Psycholinguistics
• Computational Linguistics
• Cognitive Science

Background:

• Psycholinguistic research relies on word ratings (e.g., age of acquisition, concreteness).
• Existing rating datasets cover only a fraction of vocabulary, necessitating extrapolation methods.
• Corpus-based semantic spaces and machine learning offer a solution for expanding rating coverage.

Purpose of the Study:

• To systematically compare machine learning techniques for extrapolating psycholinguistic ratings.
• To evaluate the accuracy and utility of different semantic spaces and extrapolation algorithms.
• To assess the impact of extrapolated ratings on explaining human performance in cognitive tasks.

Main Methods:

• Comparison of k-nearest neighbors and random forest extrapolation techniques.
• Utilized semantic spaces from latent semantic analysis, topic models, HAL-like models, and skip-gram models.
• Evaluated prediction accuracy and ability to incorporate additional predictors.

Main Results:

• A k-nearest neighbors variant with skip-gram word vectors yielded the most accurate predictions.
• Random forest offered flexibility in incorporating additional predictors.
• Extrapolated ratings were assessed for their ability to explain lexical decision task performance and category assignment.

Conclusions:

• While some extrapolation methods show accuracy, they may introduce data artifacts.
• Extrapolated ratings can potentially lead to different research conclusions compared to human ratings.
• The practical utility of current extrapolation methods for psycholinguistic research may be limited.