System alignment supports cross-domain learning and zero-shot generalisation.

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Psychology

Background:

• Conceptual relationships often extend across different sensory modalities, such as language and vision.
• Similarity structures between concepts in one domain may signal corresponding structures in another.
• These cross-modal similarities could aid in learning to map between domains, like object naming.

Purpose of the Study:

• To investigate whether aligning representational systems across modalities enhances learning.
• To determine if cross-modal alignment facilitates generalization to new associations.
• To model the cognitive mechanisms underlying cross-modal learning and alignment.

Main Methods:

• A paired-associate learning experiment where participants mapped visual object features to spatial locations.
• Manipulation of the alignment between the visual feature system and the spatial dimension system.
• Fitting computational models to participant data to identify the best explanatory mechanisms.

Main Results:

• Participants learned more efficiently when the visual and spatial systems were aligned.
• Aligned systems significantly improved zero-shot generalization performance.
• Computational models incorporating an unsupervised, offline alignment mechanism best explained human learning patterns.

Conclusions:

• Humans actively align entire representational systems across modalities to accelerate learning.
• This alignment process is beneficial even when learning arbitrary associations.
• Cross-modal representational alignment is a fundamental mechanism for efficient learning and generalization.