Grid Convolution for 3D Human Pose Estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• 3D human pose estimation from 2D keypoints is crucial for human-centered computer vision.
• Existing methods face challenges in accurately representing and learning from 2D pose data for 3D lifting.

Purpose of the Study:

• To develop a novel grid representation learning paradigm for improved 3D human pose estimation.
• To introduce GridConv and Semantic Grid Transformation (SGT) for mapping 2D poses to a regular grid structure.
• To enhance GridConv with attention mechanisms and develop spatial-temporal networks for video data.

Main Methods:

• Formulated a grid representation learning paradigm using GridConv based on Semantic Grid Transformation (SGT).
• Implemented SGT using both handcrafted and learnable approaches, with the learnable version showing superior performance.
• Introduced an attention module to enhance GridConv's contextual encoding capabilities.
• Developed spatial and spatial-temporal grid lifting networks for single-frame and video inputs, respectively.

Main Results:

• The proposed grid lifting network significantly outperforms existing methods on Human3.6M and MPI-INF-3DHP datasets.
• Both handcrafted and learnable SGT methods achieved promising results, with learnable SGT demonstrating better performance.
• The enhanced GridConv with attention improved the encoding of contextual cues.
• Spatial-temporal networks effectively captured spatial and temporal joint variations.

Conclusions:

• The novel grid representation learning paradigm using GridConv shows great potential for 3D human pose estimation.
• The proposed methods achieve state-of-the-art results and demonstrate strong generalization capabilities.
• The approach is effective across various keypoint-based tasks, including 3D hand pose estimation, head pose estimation, and action recognition.