Collaborative Learning and Style-Adaptive Pooling Network for Perceptual Evaluation of Arbitrary Style Transfer

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Arbitrary Style Transfer (AST) research has advanced, but perceptual evaluation of AST images remains challenging.
• Existing methods use hand-crafted features and simple pooling, failing to account for complex factors like structure preservation, style similarity, and overall vision (OV).
• Simple pooling strategies struggle with varying importance weights between quality factors and final perceived quality.

Purpose of the Study:

• To propose a novel learnable network, the collaborative learning and style-adaptive pooling network (CLSAP-Net), for improved perceptual evaluation of AST images.
• To address the limitations of existing methods in handling complex quality factors and their varying importance in AST image quality assessment (IQA).
• To develop a style-adaptive pooling strategy that dynamically adjusts the importance weights of quality factors based on style type.

Main Methods:

• Introduced CLSAP-Net with three components: content preservation estimation network (CPE-Net), style resemblance estimation network (SRE-Net), and OV target network (OVT-Net).
• Utilized self-attention mechanisms and joint regression in CPE-Net and SRE-Net to generate quality factors and weighting vectors.
• Employed a style-adaptive pooling strategy in OVT-Net, guided by style type, to collaboratively learn final quality by adjusting importance weights.

Main Results:

• The proposed CLSAP-Net demonstrated effectiveness and robustness in extensive experiments on AST IQA databases.
• The style-adaptive pooling strategy allows for self-adaptive quality pooling by understanding style types.
• The network successfully addresses the limitations of simple quality pooling in AST image evaluation.

Conclusions:

• CLSAP-Net offers a significant improvement in the perceptual evaluation of arbitrary style transfer images.
• The novel style-adaptive pooling mechanism enhances the accuracy and adaptability of image quality assessment.
• The developed approach provides a more sophisticated method for understanding and quantifying the perceived quality of stylized images.