Audiovisual correspondence between musical timbre and visual shapes

Area of Science:

• Cross-modal perception
• Psychoacoustics
• Visual perception

Background:

• Previous audio-visual studies often used simple stimuli like pure tones.
• Research has explored correspondences between features like pitch and loudness with visual attributes.
• Limited research exists on the relationship between complex musical timbre and visual shape.

Purpose of the Study:

• To investigate cross-modal correspondences between musical timbre and shapes using complex sounds.
• To examine associations between normalized timbre and visual shape, color, and position.
• To determine if color or grayscale influences timbre-shape associations.

Main Methods:

• Utilized 23 musical sounds varying in fundamental frequency and timbre, with fixed loudness.
• Presented sounds against colored and grayscale shapes, with participants selecting visual equivalents.
• Conducted an online experiment with 119 participants, including musicians and non-musicians, some with synesthesia-like experiences.

Main Results:

• A strong association was found between normalized timbre and visual shapes.
• Soft timbres were linked to rounded shapes (blue, green, light gray).
• Harsh timbres were linked to angular shapes (red, yellow, dark gray); color/grayscale had no effect.
• Fundamental frequency did not correlate with height, grayscale, or color.

Conclusions:

• Musical timbre is significantly associated with perceived shape.
• These findings support the development of sensory substitution systems, potentially aiding the blind in shape perception.
• The timbre-shape correspondence is robust and independent of color or grayscale variations.