Emotional Semantics-Preserved and Feature-Aligned CycleGAN for Visual Emotion Adaptation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Deep neural networks (DNNs) excel in vision tasks but struggle with domain shift.
• Unsupervised domain adaptation (UDA) addresses transferring knowledge from labeled source domains to unlabeled target domains.

Purpose of the Study:

• To develop an effective UDA method for visual emotion analysis.
• To enhance both emotion distribution learning and dominant emotion classification.

Main Methods:

• Proposed CycleEmotionGAN++, an end-to-end cycle-consistent adversarial model.
• Improved CycleGAN with multiscale structured cycle-consistency for pixel-level domain alignment.
• Introduced dynamic emotional semantic consistency loss to preserve source image emotion labels.
• Implemented feature-level alignment between adapted and target domains for classifier training.

Main Results:

• CycleEmotionGAN++ demonstrated significant improvements over state-of-the-art UDA approaches.
• Achieved strong performance on Flickr-LDL, Twitter-LDL, ArtPhoto, Flickr, and Instagram datasets.
• Effectively addressed domain shift challenges in visual emotion analysis.

Conclusions:

• CycleEmotionGAN++ offers a robust solution for UDA in visual emotion analysis.
• The proposed methods enhance model generalizability and performance on unlabeled target domains.