Unsupervised Sequence Alignment between Video and Human Center of Pressure.

Area of Science:

• Biomechanics
• Computer Vision
• Machine Learning

Background:

• Accurate center of pressure (COP) estimation from video relies on precise synchronization with human pose data.
• Misaligned video and COP sequences lead to significant errors in supervised learning tasks.
• Existing methods struggle with unsynchronized datasets featuring different start times and frame rates.

Purpose of the Study:

• To develop an unsupervised deep learning approach for aligning video and COP sequences.
• To improve the accuracy of COP estimation in downstream tasks using synchronized data.
• To address challenges posed by unsynchronized datasets in real-world applications.

Main Methods:

• Trained an Alignment Network using unsupervised deep learning on a synchronized dataset.
• Applied the Alignment Network to an unsynchronized dataset to correct temporal offsets.
• Developed a Differential Network to estimate COP sway level on the aligned, unsynchronized data.

Main Results:

• The Alignment Network successfully removed 84.4% of the temporal offset on the synchronized dataset.
• The proposed method demonstrated over 20% improvement in COP sway level estimation on the unsynchronized dataset.
• This approach significantly enhances precision for downstream tasks involving video and COP data.

Conclusions:

• Unsupervised deep learning effectively aligns video and COP sequences, even with temporal misalignments.
• The developed Alignment and Differential Networks offer a robust solution for accurate COP estimation.
• This method has practical implications for improving human movement analysis and related fields.