Early Action Prediction by Soft Regression.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Early action prediction is crucial for human-computer interaction and surveillance.
• Existing methods often require known sequence progress, limiting real-world applicability.
• Depth cameras offer rich spatial information for action recognition.

Purpose of the Study:

• To develop a novel soft regression framework for early action prediction using low-cost depth cameras.
• To address the ambiguity and uncertainty inherent in predicting actions from incomplete sequences.
• To improve the accuracy and robustness of early action prediction systems.

Main Methods:

• Proposed a soft regression-based early prediction framework estimating soft labels for subsequences at different progress levels.
• Developed a Multiple Soft labels Recurrent Neural Network (MSRNN) considering inter-subsequence relationships and inter-class label discrepancies.
• Introduced a local accumulative frame feature (LAFF) for efficient real-time RGB-D feature computation.

Main Results:

• The proposed soft regression model significantly outperforms existing early action prediction models on benchmark datasets.
• Early action prediction using RGB-D sequences demonstrated higher accuracy compared to RGB-only sequences.
• The LAFF feature enabled efficient real-time performance.

Conclusions:

• The novel soft regression framework effectively resolves ambiguity in early action prediction.
• The MSRNN model enhances prediction accuracy by considering complex label relationships.
• RGB-D data provides a significant advantage for accurate and robust early action prediction.