Reinforcement learning coupled with finite element modeling for facial motion learning.

Area of Science:

• Computational biomechanics
• Machine learning in healthcare
• Facial animation and modeling

Background:

• Facial palsy and transplantation impair facial motion due to muscle/nerve issues.
• Existing computer-aided and physics-based models have limited predictive capacity for complex facial movements.
• Emerging properties of facial motion require advanced predictive solutions.

Purpose of the Study:

• To couple reinforcement learning (RL) with finite element modeling (FEM) for facial motion learning and prediction.
• To develop a novel workflow for integrating FEM simulations into RL for facial motion analysis.
• To explore muscle excitation patterns for improved facial motion prediction.

Main Methods:

• Developed a novel workflow linking RL and rigid multi-body dynamics via an information exchange protocol.
• Utilized a physically-based face model in the Artisynth platform.
• Employed deep deterministic policy gradient (DDPG) and Twin-delayed DDPG (TD3) algorithms for symmetry and smile movement simulations, validated against the Bosphorus database.

Main Results:

• The RL agent learned optimal policies after over 300 training episodes, improving symmetry-oriented motion reward by ~89% (from -2.06 to -0.23).
• Simulated smile movements showed mouth corner elevation of 0.35 cm, consistent with database observations (0.4 ± 0.32 cm).
• Demonstrated successful integration of FEM simulations within an RL framework for facial motion.

Conclusions:

• Coupling RL with FEM provides a powerful approach for learning and predicting facial motion patterns.
• This novel integration scheme enables exploration of muscle excitation for realistic facial movements.
• The developed workflow holds promise for guiding and optimizing rehabilitation programs for facial palsy and transplantation patients.