Learning about color from language

Area of Science:

• Cognitive Science
• Linguistics
• Computational Linguistics

Background:

• Color-adjective associations are often linked to visual experiences (e.g., red=hot).
• Congenitally blind individuals exhibit similar color associations, suggesting non-visual learning mechanisms.
• Language plays a crucial role in shaping abstract concepts and associations.

Purpose of the Study:

• To investigate whether color-adjective associations are embedded in the statistical structure of language.
• To determine if these language-based associations align with those reported by blind and sighted individuals.
• To identify the linguistic patterns most influential in forming these associations.

Main Methods:

• Utilized word embeddings trained on large English language corpora (spoken, written, and fiction).
• Applied a projection method to word embeddings to extract color-adjective associations.
• Compared model-derived associations with associations reported by human participants (blind and sighted).
• Analyzed sentence structures within corpora to pinpoint associative learning mechanisms.

Main Results:

• Language embeddings accurately predicted human color-adjective associations, including those of the congenitally blind.
• Embeddings from a fiction corpus yielded the most predictive associations, outperforming GPT-4.
• Models learn these associations through indirect (second-order) word co-occurrences.
• Humans can identify specific words that inform color-adjective associations when prompted.

Conclusions:

• Color-adjective associations are robustly represented in the statistical properties of language.
• Linguistic co-occurrences provide a mechanism for aligning word meanings across individuals with diverse perceptual experiences.
• Fiction corpora are particularly effective in capturing nuanced semantic associations.