ShaTure: Shape and Texture Deformation for Human Pose and Attribute Transfer.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Synthesis

Background:

• Pose-guided person image synthesis faces challenges in maintaining shape and texture quality due to spatial misalignments from occlusions and multi-viewpoints.
• Existing methods struggle to generate high-fidelity textures in controllable pose transfer tasks, often neglecting shape and texture deformation.

Purpose of the Study:

• To present a novel end-to-end pose transfer framework for transforming source person images to arbitrary poses with controllable attributes.
• To address the limitations of existing methods in generating high-fidelity texture and maintaining shape integrity during pose transfer.

Main Methods:

• Developed a new image reconstruction decoder, ShaTure, which formulates shape and texture in a braiding manner for mutual fine-tuning at feature and pixel levels.
• Introduced an Adaptive Style Selector (AdaSS) Module to enhance multi-scale feature extraction through channel-wise attention and self-recalibration of feature maps.

Main Results:

• The proposed framework demonstrates superiority over state-of-the-art methods in human pose and attribute transfer tasks, validated by quantitative and qualitative assessments.
• Ablation studies confirm the effectiveness and robustness of individual contributions, highlighting the efficacy of the ShaTure decoder and AdaSS module.

Conclusions:

• The novel framework effectively tackles challenges in pose-guided person image synthesis, achieving high-fidelity texture and shape representation.
• The ShaTure decoder and AdaSS module represent significant advancements in controllable pose transfer, offering improved performance and robustness.