Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Simulation

Background:

• Agent-based synthetic crowd simulation enables cost-effective, large-scale animation of digital humans.
• Previous models relied on static assumptions, limited datasets, or homogeneous policies, restricting generalization.
• Reinforcement learning (RL) has been applied to navigation policies but often results in static, homogeneous rules with limited scenario generalization.

Purpose of the Study:

• To develop a flexible, multi-agent reinforcement learning approach for synthetic crowd simulation.
• To enable the creation of heterogeneous synthetic crowds with generalized policies.
• To achieve comparable computational performance to traditional methods while improving emergent behaviors and movement quality.

Main Methods:

• A multi-agent reinforcement learning (MARL) approach was used to learn a parametric predictive collision avoidance and steering policy.
• Training was conducted over a parameter space to ensure model flexibility across various crowd configurations.
• A model-free approach was employed, avoiding centralization of internal agent information, control signals, or communication.

Main Results:

• The goal-conditioned approach learned a parametric policy that supports heterogeneous synthetic crowds.
• The model demonstrated policy generalization across unseen scenarios, agent parameters, and out-of-distribution parameterizations.
• The learned model achieved comparable computational performance to traditional methods and produced emergent behaviors like laminar flow, shuffling, bottleneck, and side-stepping.

Conclusions:

• The proposed MARL approach offers a flexible and generalizable method for synthetic crowd simulation.
• The model effectively generates heterogeneous crowds with emergent behaviors and high movement quality.
• This approach advances the usability and realism of synthetic crowd domains compared to prior methods.