Compositional Learning of Human Activities With a Self-Organizing Neural Architecture

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Assistive systems require recognizing human activities and their constituent actions.
• Current methods often separate action and activity recognition, leading to sensitivity in temporal segmentation.
• Learning activity compositionality is crucial for human-robot interaction.

Purpose of the Study:

• To develop a novel approach for jointly learning human activities at semantic and temporal levels.
• To enable recognition of both transitive actions (e.g., reaching) and high-level activities (e.g., having breakfast).
• To achieve unsupervised semantic segmentation of activities into composing actions.

Main Methods:

• A hierarchical network of GWR (Growing When Required) networks processes visual cues from skeletal data and object interactions.
• The architecture learns spatiotemporal dependencies from RGB-D sequences.
• A novel top-down modulation mechanism constrains learning using action and activity labels.

Main Results:

• The proposed model outperforms state-of-the-art methods in high-level activity classification.
• The architecture successfully segments high-level activities into composing actions without supervision.
• The top-down modulation mechanism controls neural growth without performance degradation.

Conclusions:

• The novel approach effectively learns human activity compositionality, advancing assistive systems and robot companions.
• Joint learning of actions and activities offers a more robust method than separate recognition tasks.
• The unsupervised segmentation and modulation mechanism provide flexible and efficient learning.