Force from Motion: Decoding Control Force of Activity in a First-Person Video.

Area of Science:

• Computer Vision
• Robotics
• Physics

Background:

• First-person videos capture actor's physical interactions.
• Estimating forces and torques from these videos is challenging.
• Existing methods struggle with dynamic, real-world activities.

Purpose of the Study:

• To develop a method for estimating active forces and torques from first-person videos.
• To leverage ego-motion and visual semantics for physical state reconstruction.
• To enable force estimation in scenarios lacking inertial measurement units.

Main Methods:

• Utilized a dynamical system to model actor's physical state transitions.
• Reconstructed 3D camera trajectory and learned scale and gravity from ego-motion and visual cues.
• Employed optimal control theory by minimizing reprojection error to solve the system.

Main Results:

• Achieved quantitatively equivalent reconstruction to IMU measurements for gravity and scale.
• Outperformed 2D optical flow methods in active action recognition.
• Successfully applied to diverse activities like mountain biking, skiing, and wingsuit flying.

Conclusions:

• The proposed method effectively estimates forces and torques from first-person videos.
• It provides a viable alternative for physical state estimation when IMUs are unavailable.
• This opens possibilities for analyzing human dynamics in unconstrained environments.