Bridging Global Context Interactions for High-Fidelity Pluralistic Image Completion.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Image completion is a challenging task requiring high fidelity and diversity.
• Existing autoregressive methods are computationally expensive, limiting inference speed.

Purpose of the Study:

• Introduce PICFormer, a novel Transformer-based framework for Pluralistic Image Completion.
• Achieve high quality and diversity in image completion with significantly faster inference speeds.

Main Methods:

• Utilize code-shared codebook learning with a restrictive CNN for local visible token representation.
• Employ a Transformer architecture for efficient global context modeling.
• Propose a novel attention-aware layer (AAL) to enhance appearance consistency.

Main Results:

• PICFormer learns semantically-rich discrete codes, improving image quality.
• Simultaneous token sampling leads to over 100x faster inference compared to autoregressive methods.
• The framework surpasses State-of-the-Art methods on multiple image completion benchmarks.

Conclusions:

• PICFormer offers an efficient and effective solution for Pluralistic Image Completion.
• The proposed methods enhance both the quality and diversity of generated images while drastically reducing inference time.