Multi-Dataset, Multitask Learning of Egocentric Vision Tasks.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Egocentric vision tasks like action recognition suffer from limited labeled data, increasing the risk of model overfitting.
• Existing methods often rely on single datasets, which have differing label sets, further restricting training data availability.

Purpose of the Study:

• To address the scarcity of labeled data in egocentric vision by developing a novel multitask learning scheme.
• To enhance action recognition performance by leveraging related tasks and multiple datasets with diverse label sets.

Main Methods:

• Implemented a multitask learning framework incorporating related auxiliary tasks (e.g., object presence, hand position) to guide the network.
• Extended the multitask paradigm to include and effectively mix data from multiple datasets with heterogeneous label sets during training.

Main Results:

• Demonstrated significant improvements in egocentric action recognition across multiple benchmark datasets (EPIC-Kitchens, EGTEA Gaze+, ADL, Charades-EGO) compared to single-dataset baselines.
• Achieved state-of-the-art performance on the EGTEA dataset, surpassing previous results by 2.47%.

Conclusions:

• The proposed multitask learning approach effectively mitigates data scarcity issues in egocentric vision.
• The method successfully leverages cross-dataset task correlations, leading to more robust and accurate action recognition models.