Parts2Whole: Generalizable Multi-Part Portrait Customization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Generation

Background:

• Multi-part portrait customization is crucial for digital human creation, enabling realistic image generation from specified body parts.
• Current methods face limitations: test-time fine-tuning is slow and struggles with composition, while feed-forward methods lack fine-grained control.

Purpose of the Study:

• To introduce Parts2Whole, a diffusion-based generator for harmoniously integrating multiple reference parts into high-fidelity human images.
• To overcome the limitations of existing portrait customization techniques by offering both efficiency and precise control.

Main Methods:

• Proposed a multi-reference mechanism within a diffusion model for seamless part integration.
• Developed a detail-aware appearance encoder, leveraging pre-trained U-Net priors for rich feature extraction from reference images.
• Integrated pose map conditioning and a shared self-attention mechanism with mask information for precise part selection and posture control.

Main Results:

• Parts2Whole successfully generates high-fidelity human images by harmoniously integrating multiple specified body parts.
• The approach demonstrates superior performance compared to existing methods in portrait customization.
• Validated applicability to related tasks, including pose transfer and pose-guided human image generation.

Conclusions:

• Parts2Whole offers a versatile and effective solution for multi-part portrait customization, addressing limitations of prior work.
• The method provides precise control over appearance and pose, advancing digital human creation and related image generation tasks.