CycleACR: Cycle Modeling of Actor-Context Relations for Video Action Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video action detection is challenging due to complex actor interactions and scene context.
• Existing methods struggle with scene variations and background interference in relation modeling.
• Effective modeling of actor-scene relationships is crucial for accurate action recognition.

Purpose of the Study:

• To propose a novel network, Cycle Actor-Context Relation (CycleACR), for improved video action detection.
• To enhance relation modeling by selecting actor-related scene context instead of raw video data.
• To achieve state-of-the-art performance in video action detection.

Main Methods:

• Developed a Cycle Actor-Context Relation (CycleACR) network with a symmetric graph for bidirectional actor-context relation modeling.
• Introduced Actor-to-Context Reorganization (A2C-R) and Context-to-Actor Enhancement (C2A-E) modules.
• Incorporated parallel local/global temporal context modeling and a context-aware memory bank.

Main Results:

• Achieved state-of-the-art performance on AVA (40.6 mAP) and UCF101-24 (84.7 mAP) datasets.
• Demonstrated the effectiveness of the proposed A2C-R module for relation modeling.
• Ablation studies and visualizations confirmed the improvements from cycle actor-context relation modeling.

Conclusions:

• The proposed CycleACR network significantly advances video action detection through effective actor-context relation modeling.
• The novel A2C-R module is key to improving context feature reorganization and actor feature enhancement.
• The method offers a robust approach for capturing high-order relations and temporal dependencies in video data.