Verbal-Person Nets: Pose-Guided Multi-Granularity Language-to-Person Generation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Personalized image editing using natural language is user-friendly but requires handling varying text-visual relevance.
• Synthesizing realistic person images from descriptions presents challenges in capturing both global structure and fine-grained details.

Purpose of the Study:

• To propose a novel pose-guided multi-granularity attention architecture for end-to-end person image generation from text.
• To improve the accuracy and realism of generated person images by addressing semantic relevance at different granularities.

Main Methods:

• A U-Net architecture incorporating sentence-level descriptions and pose features for coarse image generation.
• A conditional generative adversarial network (GAN) framework for realistic image synthesis.
• Two-stream discriminators: text-relevant local discriminators for fine-grained details and a global discriminator with pose-weighting for overall coherence.

Main Results:

• The proposed model effectively synthesizes person images conditioned on natural language descriptions and pose information.
• The multi-granularity attention mechanism enhances the capture of both global outlines and specific body part details.
• Experimental results demonstrate the superiority of the proposed method over existing approaches for person image generation.

Conclusions:

• The novel pose-guided multi-granularity attention architecture offers a robust solution for text-to-image synthesis of persons.
• The method successfully integrates textual semantics with pose information for high-fidelity person image generation.
• This work advances the state-of-the-art in personalized image editing and realistic human image synthesis.