通过深度强化学习超越了Gran Turismo的冠军车手
Peter R Wurman1, Samuel Barrett2, Kenta Kawamoto3
1Sony AI, New York, NY, USA. peter.wurman@sony.com.
Nature
|February 10, 2022
概括
研究人员为Gran Turismo赛车模拟开发了人工智能 (AI) 代理,可以与顶级人类电竞车手竞争. 这种人工智能结合了速度和战术, 在复杂的实时环境中展示了先进的控制.
科学领域:
- 人工智能
- 机器人技术
- 计算神经科学
背景情况:
- 在与人类互动的物理系统中实时决策存在重大挑战.
- 汽车赛事就是这些挑战的典范,需要复杂的战术机动和精确的车辆控制.
- 像Gran Turismo这样的赛车模拟精确地模拟了这些复杂的多个代理的动态.
研究的目的:
- 培养能够在Gran Turismo赛车模拟中竞争的人工智能 (AI) 代理人.
- 为人工智能赛车手制定一个综合控制政策, 结合速度和战术决策.
- 在模拟规则中创造一种奖励功能,
主要方法:
- 使用最先进的无模型深度强化学习算法.
- 实施混合场景培训,使代理人接触到不同的比赛条件.
- 开发了一种专门的奖励功能,以平衡表现与遵守体育规范.
主要成果:
- 训练有素的人工智能代理, 命名为Gran Turismo Sophy, 展示了与世界级电竞车手竞争的能力.
- 人工智能特工表现出极高的速度和复杂的战术机动.
- 苏菲成功地赢得了对抗四名顶级Gran Turismo玩家的比赛.
结论:
- 这项研究展示了深度强化学习在人与人交互领域控制复杂动态系统的潜力.
- 人工智能代理可以在需要速度和战略互动的模拟环境中实现冠军级别的性能.
- 在不精确的人类规范领域控制人工智能,如体育精神,既带来了可能性,也带来了挑战.
相关概念视频
01:23Reinforcement
411
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
411
01:18Hierarchy of Motor Control
3.9K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
3.9K
01:17Rolling Resistance: Problem Solving
486
Rolling resistance, also known as rolling friction, is the force that resists the motion of a rolling object, such as a wheel, tire, or ball, when it moves over a surface. It is caused by the deformation of the object and the surface in contact with each other, as well as other factors like internal friction, hysteresis, and energy losses within the materials. Rolling resistance opposes the object's motion, requiring additional energy to overcome it and maintain movement. In practical...
486
01:10Torque
16.0K
Torque is an important quantity for describing the dynamics of a rotating rigid body. We see the application of torque in many ways in the world, such as when pressing the accelerator in a car, which causes the engine to apply additional torque on the drivetrain. Here, we define torque and provide a framework to create an equation to calculate torque for a rigid body with fixed-axis rotation.
Torque can be considered as the rotational counterpart to force. Since forces change the translational...
Torque can be considered as the rotational counterpart to force. Since forces change the translational...
16.0K
01:12Observational Learning
360
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
360
01:07End Point Prediction: Gran Plot
668
A Gran plot is used to predict the equivalence volume or endpoint of a potentiometric or acid-base titration without reaching the endpoint. Typically, titration data is collected as a function of the titrant's volume up to a point less than the equivalence volume and then transformed into a linear format. The straight line is extended to the x-axis, indicating the necessary titrant volume to achieve the equivalence point.
For potentiometric titration, the Gran plot is created by plotting...
For potentiometric titration, the Gran plot is created by plotting...
668