HAKE: A Knowledge Engine Foundation for Human Activity Understanding.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human activity understanding is crucial for AI applications in healthcare and behavior analysis.
• Current deep learning methods struggle with direct pixel-to-semantics mapping due to the complexity of activity patterns compared to object recognition.
• Existing approaches face limitations in generalization and performance on challenging benchmarks.

Purpose of the Study:

• To propose a novel two-stage paradigm for human activity understanding.
• To develop a framework that maps pixels to an intermediate space of atomic activity primitives.
• To infer semantic meaning using interpretable logic rules applied to detected primitives.

Main Methods:

• Developed a two-stage framework: pixel-to-primitive mapping and primitive-to-semantics inference.
• Constructed a knowledge base with over 26 million primitive labels and logic rules.
• Utilized human priors and automatic discovery for building the primitive space and rules.
• Implemented the Human Activity Knowledge Engine (HAKE) framework.

Main Results:

• The HAKE framework demonstrated superior generalization ability compared to canonical methods.
• Achieved enhanced performance on challenging human activity understanding benchmarks.
• The two-stage approach effectively addresses the limitations of direct pixel-to-semantics mapping.

Conclusions:

• The proposed paradigm offers a more effective approach to human activity understanding.
• HAKE provides a robust solution with improved performance and generalization.
• The framework's ability to use interpretable logic rules enhances semantic inference.