Deep reinforcement learning for weak human activity localization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional human activity localization methods require extensive frame-by-frame annotations, which are costly and time-consuming.
• The large search space in 3D activity localization necessitates generating numerous proposals, increasing computational complexity.

Purpose of the Study:

• To develop an efficient human activity localization method that overcomes limitations of existing approaches.
• To reduce the need for detailed annotations and decrease the number of required proposals.

Main Methods:

• A unified deep Q-network with weak reward and weak loss (DWRLQN) was proposed.
• Incorporated weak knowledge and constraints on temporal dynamics within a deep reinforcement learning framework.
• Utilized sparse spatial supervision, assuming only partial frame annotations.

Main Results:

• The DWRLQN model demonstrated promising performance on UCF-Sports, UCF-101, and sub-JHMDB datasets.
• Achieved high accuracy with a significantly reduced number of proposals.
• Outperformed fully supervised methods when trained with partial annotations and weak information.

Conclusions:

• The proposed DWRLQN offers an effective solution for human activity localization with reduced annotation requirements.
• Weak supervision and reinforcement learning provide a powerful paradigm for video analysis tasks.
• This approach significantly advances the efficiency and practicality of activity localization in large video datasets.