Shared sensitivity to data distribution during learning in humans and transformer networks.

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Machine Learning

Background:

• Understanding human learning mechanisms is crucial for advancing AI.
• Transformer networks are state-of-the-art deep learning models.
• Comparing human and AI learning can reveal fundamental principles.

Purpose of the Study:

• To investigate whether humans learn similarly to transformer networks.
• To compare 'in-context' and 'in-weights' learning in both humans and AI.
• To examine the impact of training data distribution on learning strategies.

Main Methods:

• Trained 530 humans and transformer networks on a rule learning task.
• Manipulated training data diversity and redundancy.
• Measured generalization to novel queries ('in-context' learning) and memory recall ('in-weights' learning).

Main Results:

• Humans and transformers showed similar responses to training data manipulations.
• Redundancy and diversity traded off in driving 'in-weights' and 'in-context' learning, respectively.
• A balanced mix of diversity and redundancy enabled tandem learning strategies in both.

Conclusions:

• Data-distributional properties impacting learning are conserved across humans and transformers.
• Humans, unlike transformers, benefit from dynamic training curricula emphasizing early diversity.
• While core learning mechanisms show parallels, human learning exhibits unique curriculum sensitivity.