Cross-Situational Word Learning With Multimodal Neural Networks

Area of Science:

• Cognitive Science
• Computational Linguistics
• Artificial Intelligence

Background:

• Cross-situational word learning involves integrating ambiguous scene-utterance pairs to map words to referents.
• Multimodal neural networks in machine learning excel at visual-linguistic tasks using raw data.
• These networks offer a novel approach to modeling cognitive processes in word acquisition.

Purpose of the Study:

• To investigate if recent machine learning multimodal neural networks can model cross-situational word learning phenomena.
• To evaluate the explanatory power of these networks against established psychological findings in word learning.

Main Methods:

• Examined two variants of a multimodal neural network architecture.
• Trained networks on cross-situational data to learn word-referent mappings.
• Assessed network performance against seven behavioral phenomena in word learning literature.

Main Results:

• Networks successfully learned word-referent mappings within a single training epoch, mirroring experimental conditions.
• The models captured several, but not all, studied cross-situational word learning phenomena.
• Failures were consistently observed in phenomena related to reasoning via the mutual exclusivity principle.

Conclusions:

• Generic neural network learning algorithms can naturally account for some aspects of cross-situational word learning.
• Additional inductive biases are necessary to explain word learning phenomena, particularly those involving mutual exclusivity.
• Machine learning models provide valuable insights into the computational mechanisms underlying human word acquisition.