A Multi-Modal Egocentric Activity Recognition Approach towards Video Domain Generalization

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Egocentric activity recognition, using wearable cameras, is challenged by complex body motions within videos.
• Existing methods often require extensive training data or complex unsupervised domain adaptation techniques to handle data discrepancies.

Purpose of the Study:

• To propose a novel, domain-generalized approach for egocentric human activity recognition.
• To develop robust models capable of accurately predicting human activities in egocentric video sequences with minimal target domain involvement.

Main Methods:

• Introduction of a novel three-stream deep neural network architecture.
• Integration of Vision Transformers and Residual Neural Networks.
• Training the network using multi-modal data with simple source domain data manipulation.

Main Results:

• Demonstrated superiority over recent state-of-the-art research works.
• Achieved robust human activity prediction in challenging egocentric video datasets.
• Showcased the effectiveness of simple data manipulation and minimal target domain involvement.

Conclusions:

• The proposed approach offers a more efficient and effective solution for domain-generalized egocentric activity recognition.
• The novel three-stream network architecture provides a robust framework for analyzing egocentric video data.
• This method advances the field by simplifying domain adaptation challenges in wearable camera-based activity recognition.