Reasoning action-centric temporal relations at rich feature hierarchies for action recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Action recognition is crucial for understanding human activities.
• Existing methods often rely on high-level semantics and background information, limiting performance.
• Temporal relational reasoning is key but challenging to implement effectively.

Purpose of the Study:

• To propose a novel Action-centric Temporal-relational Reasoning (ATR) block for improved action recognition.
• To develop an action-centric and hierarchical approach to temporal relational reasoning.
• To enhance the efficiency and effectiveness of action recognition models.

Main Methods:

• Introduced a novel Action-centric Temporal-relational Reasoning (ATR) block.
• ATR consists of an Action-related Region Locator (ARL) and an Efficient Action-centric Reasoner (EAR).
• The ATR block was integrated into existing action recognition models (TSN, TSM, SlowOnly) at various depths.

Main Results:

• Consistent and notable improvements in action recognition accuracy were observed across benchmarks (Something-Something V1, V2, Kinetics).
• The proposed ATR block demonstrated effectiveness in action-centric and hierarchical temporal relational reasoning.
• The approach achieved these gains with minor computational overhead.

Conclusions:

• The ATR block significantly enhances action recognition by focusing on action-centric temporal relations.
• The hierarchical and action-centric formulation is effective for temporal relational reasoning.
• This approach holds practical significance for real-world action recognition applications.