Related Experiment Video
Updated: Jun 25, 2025
06:17
Author Spotlight: Investigating the Effects of Mind-Body-Movement Practices on Brain Function
Published on: January 26, 2024
1.9K
Expanding horizons in reinforcement learning for curious exploration and creative planning
Dale Zhou1,2, Aaron M Bornstein2,3
1Neurobiology and Behavior, 519 Biological Sciences Quad, University of California, Irvine, CA, USA dale.zhou@uci.eduhttps://dalezhou.com.
The Behavioral and Brain Sciences
|May 21, 2024
Summary
Curiosity and creativity balance familiarity with novelty. Reinforcement learning models how information seeking and generation impact planning horizons, offering solutions for this trade-off.
Area of Science:
- Cognitive Science
- Computational Neuroscience
- Artificial Intelligence
Background:
- Curiosity and creativity involve balancing the exploitation of known information with the exploration of novel information.
- Understanding this balance is crucial for explaining complex cognitive behaviors and designing intelligent systems.
Purpose of the Study:
- To computationally model the trade-off between familiarity and novelty seeking in curiosity and creativity.
- To formally capture solutions for balancing exploration and exploitation using reinforcement learning.
Main Methods:
- Utilized a reinforcement learning framework to analyze decision-making processes.
- Formulated the value of information seeking and generation based on their effects on planning horizons.
Main Results:
- Demonstrated that the complementary effects of information seeking and generation on planning horizons can formally capture the balance between familiarity and novelty.
- Identified a range of solutions for striking this balance within the computational model.
Conclusions:
- Reinforcement learning provides a powerful lens for understanding the cognitive mechanisms underlying curiosity and creativity.
- The proposed framework offers a formal method for analyzing and potentially optimizing the balance between exploration and exploitation in intelligent agents.
Related Concept Videos
01:21Cognitive Learning
237
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
237
01:12Observational Learning
166
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
166
01:24Reinforcement Schedules
144
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
144
01:22Purposive Learning
118
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
118
01:23Reinforcement
202
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
202
01:25Creative Thinking
850
Creative thinking encompasses innovative and unconventional methods for addressing challenges, often leading to groundbreaking solutions. Instead of focusing solely on enhancing existing systems, such as increasing smartphone battery capacity, creative thinking might inspire advancements like energy-efficient batteries or processors that minimize power consumption. This multidimensional approach underscores the importance of exploring novel pathways to innovation.
Divergent thinking is the...
Divergent thinking is the...
850