Toward Deeper Emotional Reflection: Crafting Affective Image Filters With Generative Priors.

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Social media facilitates emotional expression through text and images.
• Existing methods lack the ability to translate abstract textual emotions into concrete visual representations.
• There is a need for models that can generate emotionally resonant images from text.

Purpose of the Study:

• To introduce the Affective Image Filter (AIF) task.
• To develop models capable of reflecting visually-abstract emotions from text into visually-concrete images.
• To create emotionally compelling visual content.

Main Methods:

• Introduction of the AIF dataset and model formulations.
• Development of AIF-B using a multi-modal transformer architecture.
• Proposal of AIF-D, extending AIF-B by incorporating generative priors from large-scale diffusion models for deeper emotional reflection.

Main Results:

• AIF models demonstrate superior performance in content consistency and emotional fidelity compared to state-of-the-art methods.
• Quantitative and qualitative experiments validate the effectiveness of AIF models.
• Extensive user studies confirm that AIF models are significantly more effective at evoking specific emotions.

Conclusions:

• The Affective Image Filter (AIF) task and models offer a novel approach to visually representing emotions.
• AIF-D, leveraging diffusion models, shows significant potential for generating emotionally compelling images.
• The developed models hold considerable value and potential for applications in emotionally resonant content creation.