Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Generalization, Discrimination, and Extinction01:24

Generalization, Discrimination, and Extinction

432
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
432
Cognitive Learning01:21

Cognitive Learning

219
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
219
Purposive Learning01:22

Purposive Learning

98
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
98
Associative Learning01:27

Associative Learning

288
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
288
Observational Learning01:12

Observational Learning

131
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
131
Implicit Memories01:24

Implicit Memories

99
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
99

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Inferring genotype-phenotype maps using attention models.

PNAS nexus·2026
Same author

Unifying theories in high-dimensional biophysics: approaches, challenges and opportunities.

NPJ systems biology and applications·2026
Same author

Data coarse graining can improve model performance.

Physical review. E·2026
Same author

Ultrafast Synthesis of Titanium Suboxide via Magnetic Induction Heating for Enhanced Photodynamic Activity.

Chemistry (Weinheim an der Bergstrasse, Germany)·2026
Same author

Olfactory Sensing and Navigation in Turbulent Environments.

Annual review of condensed matter physics·2026
Same author

A cross-sectional analysis of dermatology resident research productivity in the United States.

JAAD international·2025

Related Experiment Video

Updated: Jun 4, 2025

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.5K

Differential learning kinetics govern the transition from memorization to generalization during in-context learning.

Alex Nguyen1, Gautam Reddy2

  • 1Princeton Neuroscience Institute, Princeton University.

Arxiv
|December 23, 2024
PubMed
Summary

In-context learning (ICL) in transformers transitions from memorization to generalization due to learning rates, not capacity. A new scaling law predicts when this generalization occurs.

More Related Videos

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm
06:35

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm

Published on: April 28, 2016

33.9K
Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation
08:04

Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation

Published on: August 23, 2017

8.2K

Related Experiment Videos

Last Updated: Jun 4, 2025

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.5K
Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm
06:35

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm

Published on: April 28, 2016

33.9K
Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation
08:04

Using a Split-belt Treadmill to Evaluate Generalization of Human Locomotor Adaptation

Published on: August 23, 2017

8.2K

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Deep Learning

Background:

  • Transformers exhibit in-context learning (ICL), adapting to new information without weight updates.
  • ICL emerges with diverse task training, showing a sharp transition from memorization to generalization.
  • Limited network capacity was hypothesized to drive this generalization shift.

Purpose of the Study:

  • Investigate the mechanistic basis of the memorization-to-generalization transition in ICL.
  • Examine the role of sub-circuit learning dynamics in ICL.
  • Develop a theoretical framework to explain ICL phenomena.

Main Methods:

  • Utilized a small transformer model on a synthetic ICL task.
  • Combined theoretical analysis with experimental validation.
  • Analyzed the learning dynamics of distinct sub-circuits.

Main Results:

  • Identified largely independent sub-circuits for memorization and generalization.
  • Demonstrated that relative learning rates, not capacity, govern the transition.
  • Uncovered a memorization scaling law determining the generalization threshold.
  • Quantitatively explained ICL acquisition timing, solution bimodality, and sensitivity to data statistics.

Conclusions:

  • The transition to ICL generalization is driven by the differential learning rates of independent memorization and generalization sub-circuits.
  • A novel memorization scaling law provides a quantitative basis for understanding ICL emergence and behavior.
  • The developed theory offers a unified explanation for various observed ICL phenomena.