Related Experiment Videos
Coevolutionary networks of reinforcement-learning agents
Ardeshir Kianercy1, Aram Galstyan
1Information Sciences Institute, University of Southern California, Marina del Rey, California 90292, USA.
Summary
Players simultaneously adapt strategies and network ties in repeated games, leading to stable star-shaped networks without exploration. With exploration, uniform networks can become stable.
Area of Science:
- Game Theory
- Network Science
- Computational Economics
Background:
- Understanding emergent network structures in strategic interactions is crucial.
- Coevolution of strategies and network topology presents complex dynamics.
Purpose of the Study:
- To model and analyze the coevolution of strategies and network ties in repeated games.
- To characterize stable network structures and strategy profiles.
Main Methods:
- Development of a reinforcement-learning model for simultaneous strategy and network adaptation.
- Analysis using coupled replicator equations for coevolutionary dynamics.
- Comprehensive analysis of three-player games and simulations for general n-player networks.
Main Results:
- Stable equilibria in large networks predominantly feature star motifs without exploration.
- Agents converge to pure strategies in stable equilibria, irrespective of mixed strategy Nash equilibria.
- A critical exploration rate can stabilize symmetric, uniformly connected network topologies.
Conclusions:
- Network structure and player strategies mutually influence each other, leading to predictable emergent patterns.
- Exploration plays a key role in determining the final network topology.
- The model provides insights into the formation of complex networks in strategic environments.
Related Concept Videos
01:23Reinforcement
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
01:24Reinforcement Schedules
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
01:12Observational Learning
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
01:20Evolutionary Psychology
Evolutionary psychology explores the origins of human behavior and mental processes by framing them within the context of natural selection, a theory famously propounded by Charles Darwin. This field asserts that many behaviors common across human societies — ranging from instinctive fear reactions to complex social interactions — arose as evolutionary adaptations. These adaptations enhanced the survival and reproductive success of our ancestors, thereby becoming embedded in the human psyche...
01:27Associative Learning
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
01:14Avoidance Learning and Learned Helplessness
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...