Toward Efficient Action Recognition: Principal Backpropagation for Training Two-Stream Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Current backpropagation methods in two-stream networks for video action recognition process all frames, which is inefficient as actions occur in short durations.
• This exhaustive processing leads to suboptimal performance and increased computational demands.

Purpose of the Study:

• To introduce Principal Backpropagation Networks (PBNets) for more efficient training of two-stream networks in video action recognition.
• To address the inefficiency of processing all frames by developing a selective backpropagation mechanism.

Main Methods:

• A 'watch-and-choose' mechanism is proposed, featuring a dense snippet-wise temporal pooling for global characteristic discovery.
• A selection strategy using Max-rule and KL-rule identifies representative snippets for backpropagation.
• Theoretically demonstrated that backpropagation on selected subsets effectively reduces the loss for all snippets.

Main Results:

• PBNets consistently outperform state-of-the-art methods on the UCF101 and HMDB51 benchmarks.
• The proposed method achieves high performance with reduced memory and computational requirements.
• Validation of the effectiveness of Max-rule and KL-rule in snippet selection.

Conclusions:

• Principal Backpropagation Networks offer a more efficient and effective approach to video action recognition training.
• The selective backpropagation strategy significantly reduces computational overhead without compromising accuracy.
• PBNets represent a significant advancement in optimizing deep learning models for video analysis.