3D Human Pose Estimation from Monocular Video Sequences in Underwater Scenarios

Area of Science:

• Computer Vision
• Biomechanical Engineering
• Human-Computer Interaction

Background:

• Estimating 3D human pose from monocular video is challenging, especially in underwater environments due to refraction, occlusion, and poor image quality.
• Existing methods often struggle with the unique distortions and occlusions present in underwater settings, limiting their applicability in sports science and biomechanics.

Purpose of the Study:

• To develop a robust 3D human pose estimation method specifically for monocular underwater video sequences.
• To address the limitations of current pose estimation techniques in handling underwater environmental factors and data scarcity.

Main Methods:

• A two-stage framework combining 2D keypoint extraction (YOLO, HRNet) and parametric model estimation (PARE, SMPL).
• An underwater variational autoencoder (UW-VAE) to learn biomechanical priors and correct pose parameters affected by underwater conditions.
• A pipeline for generating synthetic underwater datasets to overcome annotation scarcity.

Main Results:

• The proposed method achieves significant performance improvements on the SwimXYZ synthetic dataset, outperforming land-based methods.
• Validation on real-world data shows enhanced 2D keypoint accuracy after fine-tuning with synthetic data.
• The approach successfully handles occlusions and water-induced distortions for accurate pose estimation.

Conclusions:

• This novel method offers a viable solution for accurate 3D human motion analysis in underwater environments.
• The developed UW-VAE and synthetic data generation pipeline advance research in underwater biomechanics and sports training.
• The findings pave the way for improved analysis of human movement in challenging aquatic conditions.