A new statistical model for analyzing rating scale data pertaining to word meaning

Area of Science:

• Cognitive Psychology
• Psycholinguistics
• Computational Linguistics

Background:

• The concrete-abstract categorization of words is fundamental in cognitive research.
• Traditional methods using mean values on rating scales (e.g., Concreteness, Imageability) have limitations, including sensitivity to outliers and neglect of participant-specific rating trends.
• These limitations can lead to inaccurate word classifications and the identification of non-existent differences.

Purpose of the Study:

• To introduce and evaluate Median Polish Analysis (MPA) as a superior alternative for analyzing word rating scale data.
• To compare the effectiveness of MPA against traditional sample mean analysis in classifying words.
• To investigate how different rating scales (Action, Concreteness, Imageability, Multisensory) interact with analysis methods.

Main Methods:

• Collected rating data from 80 participants on nouns and verbs using four scales: Action, Concreteness, Imageability, and Multisensory.
• Applied both traditional sample mean analysis and Median Polish Analysis (MPA), including two-way and three-way models.
• Utilized analog R ² for effect size and bootstrap 95% confidence intervals (CIs) with MPA to enhance reliability.

Main Results:

• MPA demonstrated tolerance to outliers and accounted for participant rating trends, unlike sample mean analysis.
• The Action scale, analyzed with MPA, revealed a continuum of word meaning for both nouns and verbs.
• MPA successfully differentiated between continua and dichotomies/stratified results across scales, whereas sample mean analysis produced continua regardless of the scale.

Conclusions:

• Median Polish Analysis (MPA) provides a more accurate and nuanced method for classifying words based on rating scale data.
• MPA improves word categorization by mitigating the impact of outliers and incorporating individual participant differences.
• The findings highlight the importance of employing robust statistical methods in psycholinguistic research for reliable interpretation of word properties.