MoVi: A large multi-purpose human motion and video dataset

Area of Science:

• Computer Vision
• Computer Graphics
• Biomechanics
• Human Motion Analysis

Background:

• High-quality human body shape and kinematics datasets are crucial for modeling and simulation.
• Existing datasets face challenges in combining naturalistic motion with accurate ground truth body shape and pose data.
• Different motion recording systems often optimize for either naturalistic movement or precise pose estimation.

Purpose of the Study:

• To address the limitations of current datasets by creating a multimodal resource.
• To facilitate research in human pose estimation, action recognition, motion modeling, gait analysis, and body shape reconstruction.
• To enable transfer learning through synchronized, partially overlapping data from diverse hardware systems.

Main Methods:

• Collected 9 hours of optical motion capture data.
• Acquired 17 hours of synchronized video data from 4 viewpoints using stationary and hand-held cameras.
• Gathered 6.6 hours of inertial measurement units (IMUs) data.
• Recorded data from 90 actors (60 female, 30 male) performing 21 everyday and sports actions.
• Processed motion capture data into realistic 3D human meshes.

Main Results:

• A comprehensive multimodal dataset integrating optical motion capture, video, and IMU data.
• Synchronized and partially overlapping data streams suitable for transfer learning applications.
• Processed 3D human mesh data derived from motion capture.
• A diverse collection of human actions and movements captured under naturalistic conditions.

Conclusions:

• The developed multimodal dataset provides a robust foundation for advancing research in human motion analysis and body shape modeling.
• The integration of diverse data sources and synchronized recordings enhances the utility for transfer learning.
• This resource is expected to significantly contribute to fields requiring accurate human motion and shape understanding.