Reconstruction of dance movements using reinforcement recurrent autoencoder and deep wavelet autoencoder

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Dance movement reconstruction is challenging due to complex spatiotemporal dynamics.
• Existing methods often struggle with capturing nuanced movement characteristics.

Purpose of the Study:

• To propose a novel hybrid model for accurate dance movement reconstruction using skeletal data.
• To enhance the capture of temporal and spatial features in dance movements.

Main Methods:

• Developed a hybrid model combining Reinforcement Recurrent Autoencoder (RRAE) and Deep Wavelet Autoencoder (DWAE).
• Utilized skeletal data extracted via MediaPipe Holistic algorithm from dance videos.
• Optimized model performance by experimenting with input window sizes and cost functions, using the RMSprop algorithm.

Main Results:

• The hybrid model demonstrated superior performance over traditional models (RNNs, CNNs, GANs).
• Achieved lower Dynamic Time Warping (DTW), Euclidean Distance Loss (EDL), and Frame-to-Frame Variability (FFV).
• Qualitative evaluations confirmed the generation of natural and smooth dance movements.

Conclusions:

• The proposed RRAE-DWAE hybrid model offers a significant advancement in skeletal-based dance movement reconstruction.
• The method effectively captures complex movement dynamics, leading to realistic and fluid motion synthesis.