Human-Centric Transformer for Domain Adaptive Action Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Domain adaptation in action recognition is challenging due to differing data distributions.
• Existing methods often neglect crucial human cues, relying on context that may not generalize.
• Effective transfer learning requires preserving human-specific information across domains.

Purpose of the Study:

• To develop a novel approach for domain adaptive action recognition that prioritizes human-centric cues.
• To investigate the role of human cues and human-context interaction in cross-domain action recognition.
• To improve the robustness and accuracy of action recognition models when transferring knowledge to new domains.

Main Methods:

• Proposed the Human-Centric Transformer (HCTransformer) with a decoupled human-centric learning paradigm.
• Employed a human encoder for human-aware temporal modeling to retain human cues.
• Utilized a context encoder within a Transformer-like architecture to model domain-invariant human-context interactions.

Main Results:

• Achieved state-of-the-art performance on multiple benchmarks: UCF-HMDB, Kinetics-NecDrone, and EPIC-Kitchens-UDA.
• Demonstrated the effectiveness of explicitly focusing on human cues and their interactions.
• Showcased improved domain-invariant video feature learning.

Conclusions:

• The HCTransformer effectively addresses limitations of existing methods in domain adaptive action recognition.
• Prioritizing human-centric cues leads to significant performance gains in cross-domain scenarios.
• The proposed model offers a robust solution for transferring action recognition capabilities.