Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Pose-guided person image generation and animation require significant spatial manipulation of source data.
• Convolutional Neural Networks (CNNs) inherently struggle with direct spatial transformation of input data.
• Existing methods often face limitations in achieving precise control over pose and appearance.

Purpose of the Study:

• To propose a novel framework capable of differentiable spatial transformation for person image generation and animation.
• To address the limitations of CNNs in handling spatial manipulation tasks.
• To enhance the quality and coherence of generated person images and animations.

Main Methods:

• Developed a differentiable global-flow local-attention framework.
• Estimated global flow fields between source and target poses.
• Sampled local source feature patches using content-aware local attention coefficients.
• Incorporated temporal consistency modeling for video generation.

Main Results:

• The proposed framework demonstrated efficient and accurate spatial transformation of input features.
• Superior performance was achieved in both person image generation and animation tasks compared to existing methods.
• Generated coherent videos with temporal consistency for person animation.
• Showcased applicability to novel view synthesis and face image animation.

Conclusions:

• The global-flow local-attention framework effectively overcomes CNN limitations for spatial manipulation.
• The model achieves state-of-the-art results in pose-guided person image generation and animation.
• The framework's versatility extends to other spatial transformation-intensive tasks.