Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison

Area of Science:

• Cognitive Science
• Computational Linguistics
• Psychology

Background:

• Similarity judgment is crucial for human cognition, impacting learning, decision-making, and categorization.
• Existing research offers diverse representations and metrics for similarity but lacks comprehensive comparative analysis.
• The predictive power of these methods across different semantic categories remains underexplored.

Purpose of the Study:

• To systematically compare nine vector semantic representations and seven similarity metrics for predicting human similarity judgments.
• To evaluate the effectiveness of supervised dimensional weighting in similarity functions.
• To investigate category-specific effects on similarity judgment.

Main Methods:

• A factorial design combining nine word vector representations with seven similarity metrics, creating 126 pairs.
• Testing these pairs on a novel dataset of human similarity judgments for cohyponymic words across eight categories.
• Implementing supervised learning for category-specific dimensional weighting in similarity calculations.

Main Results:

• Cosine similarity and Pearson correlation performed best among unweighted similarity functions.
• Word vectors from free association norms generally outperformed text-derived vectors.
• Models incorporating human judgments for category-specific dimension weights significantly outperformed all unweighted models.

Conclusions:

• Category-specific weighting of dimensions is essential for accurate similarity prediction, indicating strong context effects.
• Dimensional weights do not generalize well across semantic categories.
• Findings inform cognitive modeling, natural language processing, and theories of semantic representation and similarity metrics.