Large Language Models Estimate Fine-Grained Human Color-Concept Associations.

Area of Science:

• Cognitive Science
• Computer Science
• Information Visualization

Background:

• Humans consistently link abstract and concrete word meanings to colors across color space.
• This color-concept association impacts visual cognition, object recognition, and interpreting data visualizations.
• Previous theories suggested cross-modal statistical structures in experience drive these associations, but their presence and learnability were unclear.

Purpose of the Study:

• To investigate if a multimodal large language model (GPT-4) can estimate human-like color-concept association ratings.
• To determine if natural environmental data contains sufficient structure for learning these associations without strong prior constraints.

Main Methods:

• Tested GPT-4's ability to rate associations between 71 colors and various concepts (abstract and concrete).
• Compared GPT-4's ratings with human ratings across different prompting strategies.
• Evaluated information visualization palettes generated using GPT-4's ratings in an empirical user study.

Main Results:

• GPT-4's color-concept ratings showed strong correlations with human ratings.
• GPT-4 outperformed previous state-of-the-art methods in automatically estimating color-concept associations.
• Information visualization palettes derived from GPT-4's data were interpretable and, in some cases, more effective than human-rated palettes.

Conclusions:

• High-order statistical patterns between language and perception in large-scale data are sufficient for learning color-concept associations without initial constraints.
• Machine-derived color-concept associations can effectively optimize information visualizations for enhanced visual communication.