Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

Area of Science:

• Computer Vision
• Robotics
• Biometrics

Background:

• Event-based cameras offer advantages like high dynamic range and low latency for computer vision tasks.
• Existing datasets for face and facial landmark detection using event cameras are limited in size and scope, hindering research progress.
• A significant gap exists in large-scale, annotated event stream datasets for facial analysis.

Purpose of the Study:

• To introduce the first large and diverse dataset, Faces in Event Streams (FiES), for face and facial landmark detection using event-based camera data.
• To provide a valuable resource for advancing research and development in event-based facial recognition and analysis.
• To facilitate the creation of novel computer vision applications leveraging event camera technology.

Main Methods:

• The study introduces the Faces in Event Streams (FiES) dataset, comprising 689 minutes of raw event data.
• The dataset includes detailed annotations for face bounding boxes and facial landmarks.
• Twelve distinct models were trained and evaluated on the FiES dataset for face and landmark detection tasks.

Main Results:

• The newly published Faces in Event Streams dataset is the first of its kind, offering extensive annotated data for event-based facial analysis.
• Models trained on the FiES dataset achieved a mean Average Precision (mAP) score exceeding 90% for bounding box and facial landmark prediction.
• Real-time face detection using event-based cameras was successfully demonstrated with the developed models.

Conclusions:

• The Faces in Event Streams dataset addresses a critical need for large-scale annotated data in event-based computer vision.
• The high performance of the trained models demonstrates the dataset's effectiveness in advancing face and facial landmark detection.
• This work paves the way for enhanced real-time facial analysis applications utilizing event-based cameras.