Adaptive pooling with dual-stage fusion for skeleton-based action recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton-based action recognition faces challenges with existing pooling strategies due to unique skeletal structure.
• High data compactness and low redundancy in skeletal data increase the risk of accuracy degradation from information loss during pooling.

Purpose of the Study:

• To propose an Improved Graph Pooling Network (IGPN) to overcome limitations of pooling in skeleton-based action recognition.
• To enhance the effectiveness and efficiency of feature extraction from skeletal data.

Main Methods:

• Introduced a region-awareness pooling strategy utilizing structural partitioning and adaptive information weighting.
• Implemented a dual-stage fusion strategy (cross fusion and information supplement modules) to prevent discriminative information loss.
• Developed IGPN-Light (efficiency-focused) and IGPN-Heavy (accuracy-focused) as plug-and-play modules for existing graph networks.

Main Results:

• IGPN-Light achieved significant accuracy improvements while reducing FLOPs by 60-70% on the NTU-RGB+D 60 dataset.
• IGPN-Heavy further boosted performance by prioritizing accuracy.
• Demonstrated effectiveness across multiple challenging benchmarks.

Conclusions:

• The proposed IGPN effectively addresses pooling challenges in skeleton-based action recognition.
• The region-awareness pooling and dual-stage fusion strategies enhance feature representation and preserve critical information.
• IGPN offers flexible solutions (IGPN-Light and IGPN-Heavy) adaptable to different performance requirements.