Face Pose Alignment with Event Cameras

Area of Science:

• Computer Vision
• Bio-inspired Sensing

Background:

• Event cameras (ECs) provide energy efficiency, high dynamic range, and high temporal resolution.
• Face pose alignment is crucial for facial processing but challenging with dynamic motion.
• ECs offer unique advantages for motion-centric facial analysis due to their event-driven nature.

Purpose of the Study:

• To develop an efficient coarse face pose alignment method using event cameras.
• To leverage ECs' motion-dependent sensing for activity-triggered facial processing.
• To address large pose variations in facial applications with a novel alignment approach.

Main Methods:

• Creation of a dataset with extreme head rotations and varying motion intensities, annotated by humans.
• Proposal of a motion detection-based alignment approach generating activity-dependent pose-events.
• Realization of alignment using cascaded regression of extremely randomized trees.

Main Results:

• The proposed method achieved an average alignment failure rate of 2.7%.
• Performance was analyzed across varying head movement speeds, localization uncertainties, face resolutions, and predictor complexities.
• Annotator disagreement on the dataset was 1%, indicating high-quality annotations.

Conclusions:

• Event cameras show significant potential for facial applications, particularly in dynamic scenarios.
• The developed coarse alignment method demonstrates efficient and effective face pose alignment using ECs.
• The approach minimizes computation by focusing on pose changes, highlighting ECs' advantages.