Predicting Daily Activities From Egocentric Images Using Deep Learning

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Daily activity recognition is crucial for understanding human behavior and enabling assistive technologies.
• Analyzing egocentric images from wearable cameras offers a unique perspective on personal activities.
• Integrating contextual information alongside visual data can improve activity recognition accuracy.

Purpose of the Study:

• To develop and evaluate a method for predicting everyday human activities using passive egocentric wearable camera images.
• To investigate the effectiveness of deep learning techniques, specifically Convolutional Neural Networks (CNNs), for this task.
• To introduce and validate a novel 'late fusion ensemble' classification method that incorporates contextual information.

Main Methods:

• Collected a dataset of 40,103 egocentric images over six months, encompassing 19 distinct activity classes.
• Employed a Convolutional Neural Network (CNN) architecture for image classification.
• Developed and integrated a 'late fusion ensemble' method to combine visual data with contextual information (time, day of week).

Main Results:

• Achieved an overall classification accuracy of 83.07% in predicting 19 different daily activities.
• Demonstrated that the late fusion ensemble method significantly increases classification accuracy by incorporating contextual data.
• Showcased promising results with fine-tuning the classifier on just one day of data for new users.

Conclusions:

• The proposed method effectively predicts daily activities from egocentric wearable camera imagery and contextual data.
• The late fusion ensemble approach represents a significant advancement in activity recognition accuracy.
• The system shows potential for personalization and adaptation with minimal training data for new users.