Assessing Human Mobility by Constructing a Skeletal Database and Augmenting it Using a Generative Adversarial Network...

Area of Science:

• Computer Science
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Deep learning (DL) models require extensive datasets for accurate human gesture recognition.
• Acquiring comprehensive datasets for physiotherapy and human motion analysis presents challenges in data synchronization and spatial-temporal linking.

Purpose of the Study:

• To develop a novel neural network simulator for generating synthetic human gesture data.
• To augment limited real-world patient motion datasets with realistic, generated data for improved DL model training.
• To address challenges in human gesture data acquisition, including synchronization and spatio-temporal representation.

Main Methods:

• Utilized OpenPose (OP) to extract human skeletal vectors from real-world videos and photographs.
• Employed Generative Adversarial Networks (GANs) to synthesize new motion data and control movement parameters.
• Restructured skeletal model joints using Depth-First Search (DFS) to emphasize movement linkages.
• Developed a posture generator to create skeletal vectors representing human movement.

Main Results:

• Successfully generated synthetic skeletal vectors depicting human movement from limited real-world data.
• Demonstrated the capability to create a sequence of virtual coordinated human movements based on scripts.
• The proposed simulator effectively extends existing databases with synthetic data for DL applications.

Conclusions:

• The developed neural network simulator provides a viable solution for augmenting human gesture datasets.
• This approach enhances the training of deep learning models for applications in physiotherapy and gesture analysis.
• The method addresses key challenges in acquiring and processing human motion data, paving the way for more robust AI systems.