Data-driven simulator of multi-animal behavior with unknown dynamics via reinforcement learning.

Area of Science:

• Robotics and Artificial Intelligence
• Computational Biology
• Behavioral Ecology

Background:

• Advances in imitation learning enable human and animal movement replication.
• Simulating realistic multi-animal behaviors is challenging due to unknown transition models and locomotion dynamics.
• Existing methods struggle to construct simulators that reproduce trajectories and support reward-driven optimization.

Purpose of the Study:

• To introduce a data-driven simulator for multi-animal behavior.
• To address challenges in simulating complex biological systems with unknown dynamics.
• To enable reward-driven optimization and counterfactual behavior prediction.

Main Methods:

• Utilized deep reinforcement learning with counterfactual simulations for a data-driven approach.
• Estimated movement variables in reinforcement learning to handle high degrees of freedom.
• Employed a distance-based pseudo-reward to align cyber and physical states.

Main Results:

• Achieved higher reproducibility and reward acquisition compared to imitation learning and reinforcement learning.
• Validated the approach using data from artificial agents, flies, newts, and silkmoths.
• Demonstrated the capability for counterfactual behavior prediction in novel experimental settings.

Conclusions:

• The developed simulator shows significant potential for understanding complex multi-animal behaviors.
• This data-driven approach overcomes limitations of traditional mathematical models in biological simulations.
• Enables advanced analysis and prediction of animal interactions and collective dynamics.