Lagrangian Motion Magnification with Landmark-Prior and Sparse PCA for Facial Microexpressions and Micromovements

Area of Science:

• Computer Vision
• Biomedical Imaging
• Machine Learning

Background:

• Video motion magnification visualizes subtle motions, with Eulerian and Lagrangian methods.
• Facial micro-motions in high-framerate videos offer insights into physiological and affective states.
• Current methods lack detailed decomposition and unsupervised analysis of facial movements.

Purpose of the Study:

• To propose a novel Lagrangian motion magnification strategy for facial videos.
• To decompose facial motions into distinct components for selective amplification.
• To enhance the analysis of subtle facial movements for clinical applications.

Main Methods:

• Developed a Lagrangian motion magnification approach integrating facial landmark information.
• Employed unsupervised sparse Principal Component Analysis (PCA) for motion decomposition.
• Utilized a deep learning-based optical flow estimation method, retrained on a microexpression database.

Main Results:

• Successfully decomposed facial displacements into global/local movements and coherent motion components.
• Achieved selective amplification of decomposed facial motion components.
• Demonstrated potential for detailed micro-movement analysis.

Conclusions:

• The proposed method offers a novel way to analyze and visualize subtle facial motions.
• This technique has potential applications in neurocognitive assessment and medical diagnostics.
• Enhanced visualization of facial micromovements can reveal clinically relevant information.