Can Deep Learning Recognize Subtle Human Activities?

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep Learning (DL) has advanced computer vision, but common datasets contain confounding factors.
• These biases hinder accurate performance estimation and limit understanding of model generalization.
• Evaluating DL models requires addressing biases in training and testing data.

Purpose of the Study:

• Introduce a novel action classification challenge where humans excel but DL models falter.
• Demonstrate the limitations of current DL models in real-world visual understanding.
• Propose a robust methodology for creating less biased datasets and comparing human vs. AI performance.

Main Methods:

• Developed a new action classification benchmark focusing on tasks like drinking, reading, and sitting.
• Implemented state-of-the-art Deep Learning models for comparison against human performance.
• Introduced a rigorous framework to mitigate confounding factors in dataset creation and evaluation.

Main Results:

• DL models achieved accuracies of 61.7% (drinking), 62.8% (reading), and 76.8% (sitting).
• Human participants consistently scored above 90% accuracy across all three tasks.
• The proposed methods aim to reduce dataset confounds for more reliable AI evaluation.

Conclusions:

• Current Deep Learning models exhibit significant performance gaps compared to human capabilities in action classification.
• Dataset biases are a critical issue limiting the true potential and generalizability of computer vision models.
• The developed benchmark and methodology offer a path towards more robust and reliable AI evaluation.