Using EfficientNet-B7 (CNN), Variational Auto Encoder (VAE) and Siamese Twins' Networks to Evaluate Human Exercises...

Area of Science:

• Computer Vision
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Current human movement analysis methods often require manual labeling and struggle with synchronization.
• Remote healthcare and fitness applications need efficient and accurate movement assessment tools.

Purpose of the Study:

• To introduce a novel approach representing human movement as static super objects in 2D images.
• To enable automated analysis, comparison, and generation of human movements for applications like physiotherapy and fitness.
• To overcome limitations of manual labeling, start/end detection, and synchronization issues in movement analysis.

Main Methods:

• Representing human movement as static super objects in 2D images.
• Utilizing a variational autoencoder (VAE) simulator for movement generation.
• Employing an EfficientNet-B7 classifier within a Siamese twin neural network for movement verification and scoring.

Main Results:

• Demonstrated automated verification and scoring of fitness exercises.
• Showcased generation of similar human skeleton movements, addressing data scarcity for deep learning.
• Validated the versatility of the super object concept for measuring, categorizing, and generating human gestures.

Conclusions:

• The proposed static super object approach offers a versatile and efficient method for human movement analysis.
• This innovation has significant implications for remote healthcare, physiotherapy, and deep learning-based applications.
• The method eliminates manual labeling and synchronization issues, paving the way for broader adoption in human behavior analysis.