Pose Guided Person Image Generation Via Dual-Task Correlation and Affinity Learning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Generation

Background:

• Pose Guided Person Image Generation (PGPIG) aims to transform person images between poses.
• Existing methods struggle with the ill-posed nature of PGPIG and require better texture mapping supervision.
• These challenges limit the realism and detail in generated images, especially under significant pose variations.

Purpose of the Study:

• To propose a novel method, Dual-task Pose Transformer Network and Texture Affinity (DPTN-TA), to address limitations in PGPIG.
• To improve the generation of perceptually realistic person images with accurate pose transformations.
• To enhance texture mapping supervision and detail preservation during image generation.

Main Methods:

• Introduced a Dual-task Pose Transformer Network and Texture Affinity (DPTN-TA) mechanism.
• Incorporated an auxiliary source-to-source task using a Siamese structure to aid ill-posed source-to-target learning.
• Developed a Pose Transformer Module (PTM) for fine-grained mapping and texture transmission, alongside a novel texture affinity loss.

Main Results:

• DPTN-TA effectively generates perceptually realistic person images with significant pose changes.
• The method demonstrates superior performance over state-of-the-art approaches in terms of LPIPS and FID metrics.
• Successfully extended to view synthesis for other objects like faces and chairs, showcasing versatility.

Conclusions:

• DPTN-TA offers a robust solution for pose guided person image generation by tackling inherent challenges.
• The proposed dual-task learning and texture affinity mechanism significantly enhance image detail and realism.
• The model's adaptability extends beyond human figures, indicating broad applicability in computer vision tasks.