MCMNET: Multi-Scale Context Modeling Network for Temporal Action Detection.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Temporal action detection in video understanding is challenging, particularly with actions of diverse durations and complex temporal relationships.
• Existing methods struggle to capture rich temporal distributions necessary for accurate analysis of such videos.

Purpose of the Study:

• To propose a novel dual-stream model for temporal action detection that effectively models contextual information at multiple temporal scales.
• To enhance the capture of multi-scale temporal information and both long-range and short-range contexts for improved video understanding.

Main Methods:

• A dual-stream model is proposed, dividing input videos into two resolution streams.
• A Multi-Resolution Context Aggregation module captures multi-scale temporal information.
• An Information Enhancement module models long-range and short-range contexts, with outputs merged for rich temporal features.

Main Results:

• Experiments were conducted on ActivityNet-v1.3, Charades, and TSU (Toyota Smarthome Untrimmed) datasets.
• The model achieved an average mAP of 32.83% on ActivityNet-v1.3.
• The approach yielded an average mAP of 27.3% on Charades and 33.1% on TSU.

Conclusions:

• The proposed dual-stream model effectively addresses the complexities of temporal action detection, especially for videos with variable action durations.
• The integration of multi-resolution context aggregation and information enhancement modules leads to features rich in temporal information.
• The model demonstrates strong performance across multiple benchmark datasets, indicating its potential for advanced video understanding applications.