How many categories are there in crossmodal correspondences? A study based on exploratory factor analysis

Area of Science:

• Cognitive Psychology
• Psychophysics
• Crossmodal Perception

Background:

• Humans exhibit crossmodal correspondence, linking features across senses (e.g., bright light-high pitch).
• Existing research suggests various types of crossmodal correspondences exist.
• The underlying mechanisms differentiating these correspondence types remain unclear.

Purpose of the Study:

• To investigate whether different types of audiovisual crossmodal correspondences share underlying mechanisms.
• To explore the role of linguistic similarity in crossmodal correspondence.

Main Methods:

• Exploratory factor analysis was employed to analyze subjective judgments of audiovisual correspondences.
• Online experiments were conducted with Japanese adult participants (N=178 in Exp. 1, N=160 in Exp. 2).
• Participants rated the degree of correspondence between auditory and visual features.

Main Results:

• Two distinct factors emerged underlying audiovisual crossmodal correspondences.
• Factor 1 included correspondences with common linguistic terms (e.g., loudness-size, pitch-vertical position).
• Factor 2 included correspondences lacking linguistic similarities (e.g., pitch-brightness, pitch-shape).

Conclusions:

• Audiovisual crossmodal correspondences can be broadly categorized into at least two types.
• These types likely differ in their reliance on language mediation.
• Linguistic similarity appears to be a critical factor in distinguishing crossmodal correspondence mechanisms.